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PREF ACE 


This issue is devoted to presenting papers given at a symposium entitled 
Analysis of Survey Data - Issues and Methods, held at Statistics Canada on 
Thursday May 3, 1984. 

The symposium was jointly sponsored by Methodology Research Committee at 
Statistics Canada and the Laboratory for Research in Probability and 
Statistics at Carleton and Ottawa Universities. The aim of this symposium was 
to demonstrate how recent developments in the area of analysis of data from 
complex surveys could be applied to analytic studies in Statistics Canada. 

The symposium opened with remarks from the Chief Statistician, 
Martin B. Wilk, who emphasized the importance that Statistics Canada places in 
enhancing its research and development capacity and in the joint endeavours by 
the practitioners and academics on such issues. The symposium consisted of 
two sessions: - A morning session, chaired by Leslie Kish of the Institute for 
Social Research at the University of Michigan, which included contributions 
from Statistics Canada presented by D. Binder, P. Cholette, L. Heslop and 
S. Kumar, in addition to the presentation of an overview of the analysis 
issues by the Chairman. 

The afternoon session chaired by the Deputy Chief Statistician, 
Ivan P. Fellegi started with brief remarks from the chair and included papers 
from R. Fay, U.S. Bureau of the Census and W. Fuller, Iowa State University. 
The session concluded with qeneral discussion of the developments on the data 
analysis issues led by J.N.K. Rao, Carleton University. Well over 20 parti- 
cipants from various Universities and Federal and Provincial Government 
Departments attended the symposium. 

A selected bibliography on the topic compiled by the Project Team on the 


Analysis of Data from Complex Surveys is also qiven at the end. 
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ON ANALYTICAL STATISTICS FROM COMPLEX SAMPLES! 
Leslie Kish? 


I want to plead the case that an important and urqent task facing mathema- 
tical statistics consists of providing useful expressions for analytical sta- 
tistics for complex sample desiqns. I should like to describe these problems 
to mathematical statisticians who should find them interesting because they 
meet the criteria of all qood problems: they are important, unsolved and 
solvable. 

The most important and difficult problems of survey sampling still await 
adequate mathematical treatment: the textbooks are aimed almost entirely at 
producing aqood estimates of aqqregates, means and ratio means. One may also 
deal with the differences of two of these, but there is only fleeting and 
occasional reference to this problem. However, with that we come to the end 
of the statistical tools available for complex samples. 

As sampling theory developed, probability sampling has been capturing the 
field of respectable sampling practice with sample desiqns, which are often 
simultaneously economical and complex. One result has been’ an increasing 
volume of sample survey data which is of high quality and which researchers 
wish to put to more involved analytical use. But the mathematical statistics 
for doing this validly are lacking. The available analytical statistics as- 
sume independence among the selected elements: but this independence is lack- 
ing in complex sample desiqns. Thus the researcher may be forced to forego 
the analysis which he considers desirable and valuable. But if he is too 
impatient or too ignorant for that act of self-denial, he may go ahead and use 
the srs formulas he finds in books on statistics, which often result in very 
serious errors. 

I hope that mathematical statisticians will be impressed with the impor- 
tance of the unsolved problems of analytical statistics for data arising from 
complex sample designs. The lack of these is a more frequent source of qross 


mistakes than any other kind of departure from the usual assumptions. 


Qverview talk for the symposium. ae 
Leslie Kish, Institute for Social Research, The Univeristy of Michiaan. 
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These problems are important, unsolved and interesting. You may ask: are 
they solvable now? Supporting my affirmative answer are three sources of 
justification. First, we observe the great recent advances in statistical 
theory. Secondly, the rapid increases in the quantity and quality of elec- 
tronic computing machines make the time ripe for the solution of some of these 
problems. There is new interest in a general method which holds promise of 
rapid advance toward useful approximations. At the Survey Research Center we 
are now introducing this method for computing estimates of variances for 
reqression coefficients and other statistics for which formulas are not now 
available. 

It seems to me that this procedure resembles that of Alexander when he 
"solved" the Gordian knot. From a theoretical viewpoint I don't know whether 
it constitutes a solution of the problem or its avoidance. But insofar as it 
promises to give good approximations for much needed variances the practicing 
statistician will welcome its development with enthusiasm and interest. In 
this way one may obtain estimates of the confidence intervals of some analyti- 
cal statistics for which specific formulas are not now available. 

All of the above is verbatim from my talk to a joint session of the 
American Statistical Association and the Institute of Mathematical Statistics 
in 1957. Since then our situation has changed but little. Our 1957 hopes for 
that cut of the Gordian knot is now much used as BRR or Balanced Repeated 
Replications (Kish and Frankel 1970, 1974). But my moving plea for distribu- 
tion theory for doubly complex analytical statistics did not move the mathema- 
tical statisticians. I know now why not, since I am sadder and wiser now. 
First, statisticians like other scientists work not on what solutions are 
needed but on those that seem feasible at the time. (Like nuclear bombs, for 
example.) Second, distribution theory for complicated statistics for complex 
samples seems too difficult to solve. Third, the solutions would have too 
many parameters to be useful. Thus my views in (Kish 1978) and today are more 
sober: "New computational methods can give us approximate variances that 
appear satisfactory for practical purposes. However, it would be more satis- 
fying to have mathematical distribution theory for analytical statistics 
(e.q. regression coefficients) without the assumptions of independence, but 
with complex correlations between sample observations. We may hope for some 


progress, but not for qenerally useful results, because of mathematical 
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complexities, and even more because the numbers of needed parameters will 
prove too great for practical utility." 

Here follow seven important points about complex samples put boldly. They 
are not all widely known or believed, but I ask you to know, believe, use and 


teach them, as I do. 


1. The effects of complex designs must be considered separately for point 
estimates and for probability statements, like confidence intervals or tests 
of hypotheses. For point estimates we have for all sample designs consistent 
approaches to parameters from similar probability-weiqhted (H-T) estimators. 
But the probability statements like confidence intervals are highly subject to 
design effects, especially in cluster sampling. . 

a) "Statistics (means, regression coefficient, etc.) approach their 
population values as the sample size increases. 

b) The approach is generally slowed by design effects. 

c) The design effects differ for different statistics, for different 
variables and different sample designs." (Kish and Frankel, 1974). 

That paper also presents the most convincing evidence for these points: and 
evidence is widespread: e.q. (Verma et al 198M). Nevertheless two famous 
statisticians completely misstated our position in discussions of our paper: 
"Here the authors make the important observation that the confidence interval 
statements for the unknown parameter are numerically not much affected by the 
lack of independence of observations introduced by complex survey techniques 
such as stratified cluster sampling." Alas, that mistake gets quoted by other 
theoreticians who fail to read our answer of survey samplers: "They 
misunderstand completely our principal and repeated message: that confidence 
interval statements are numerically areatly affected by the lack of 
independence of observations introduced by complex survey techniques such as 
stratified cluster sampling." (Kish and Frankel, 1974). 

This misunderstanding shared by naive non-statisticians with sampling 
theorists causes troubles for us survey samplers: hence we are working on a 
clearer statement. 

zs Do we need sampling errors for analytical statistics for data from 
complex surveys? Or have a few of us been devoted to a negligible even 


trivial problem? I feel like a St. Sebastian, the target practice for the 
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slings and arrows of diverse outrageous heathen. (Mixed metaphors are better 
than fixed or random.) First come the market researchers and pollsters who 
ignore us, though some have learned to put a Y between a 2 and (paq/n). 
Second, some demographers write that with their large samples and larger mea- 
surement errors they have no time for sampling errors. Third come the mathe- 
matical psychologists, econometricians and biometricians who take their linear 
models straight from mathematical statistics, and that hurts. Fourth, even 
more hurtful are the mathematical statisticians themselves, who either forget 
that their n's do not justify their means, or they invoke IID, or they use 
some Bayesian exorcism against the spirits of the sample design. Fifth and 
worst are sampling theorists who display theorems to prove that, with com- 
pletely specified models of arbitrary superpopulations, we need not worry 
about whence or how our elements were selected, nor weight them for unequal 
selection probabilities. They even convince a few survey samplers that they 
can dwell on some Olympus with their models and not come down to earth where 
the population lives. 

From these necessarily brief remarks you notice that I am an extremist for 
several reasons: a) Desian effects for analytical statistics provide common 
evidence for imperfectly specified models for the best stratified samples: b) 
We frequently find the effects of selection weights on samples; c) Relations 
between predictor and predictand variables exist in actual individuals, and 
they in real populations, and these interact with sample designs. (I am 
developing these points in a book on Statistical Design for Social Research 
for Wiley, 1985.) 

My philosophy is consistent, but in practice I am less dogmatic. I recog- 
nize that in practice: a) it is never possible to cover completely our target 
populations, hence we must always resort to models for inference; b) proba- 
bility sampling is too costly and not feasible for. most experiments; c) 
despite lack of randomization either in selection or in treatments, we often 
blunder our way to reliable results with care, replication, design, additivity 
and a little bit of luck. 

3. Analytical statistics begin with subclasses and with their compari- 
sons. In the last three decades much useful material has been published about 
variances and design effects for subclasses. There are masses of empirical 


results and several useful quiding rules based on them (Kish 1980, Kish et 
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al. 1976, Verma et al. 1980), also some recent theory (Rust 1984, Chapter 6). 

a) Distinquish between proper domains and the more common crossclasses, on 
which we focus here. 

b) Selection probabilities are preserved for crossclasses but sample sizes 
become highly variable. 

c) Estimates of totals and means from complex samples are retained in 
ratio and conditional forms. 

d) Design effects for crossclasses tend to approach to almost 1 
proportionately as the subclass sizes per primary cluster approach 1. This 
approximate model needs care and qualification but it is preferable to all 
venerable alternatives about design effects: that it is simply 1, or some 
other constant, or the same as for the entire sample. The pooled model may be 
often better than separate and highly variable computations. 

4. Comparisons of paired means tend to have desian effects greater than 1 
but considerably less than the sum of the two variances. These reductions due 
to positive covariances (hence to a kind of additivity) have been found widely 
and reqularly for comparisons both of crossclasses and of periodic surveys 
(Kish 1965, 14.1, also the above). 

ays For complex analytical statistics several methods exploit the 
potentialities of electronic computing: Taylor linearized (delta) methods, 
including machine differentiation, Balanced Repeated Replications and 
Jackknife Repeated Replications, all have been shown to yield useful estimates 
of variance and design effects for complex samples (Kish and Frankel 1970 and 
1974; Woodruff and Causey 1978), Bootstrapping may also be added in the future 
(Rao 1984). 

Analytical statistics consistently show design effects areater than 1, 
significantly greater in every sense, but also lower than design effects for 
means. The relations of design effects between diverse coefficients and 
comparisons with those for means show some reqularities. 

For useful quidance we need not only more empirical work but also more 
results from sampling theory and model building. I am disappointed frankly 
that since our early work we have not seen more publications in theory and 
models that would be directly useful for quiding inference for actual data. 
The empirical bases of design effects are necessary, but to satisfy our 


intellectual needs for understanding we need more theory and better models. 
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Furthermore, even our practical needs remain unsatisfied with merely empirical 
design effects, because they are functions jointly of the variables, of the 
type of estimates, of the sample desiqn used and of the population basis for 
the data. That four-dimensional source of variation is too complex and we 
need theory to construct models for qreater simplicity. 

6. Categorical data analysis is an important area, rapidly developing, and 
several contributions have been made to apply these methods to complex survey 
data "(Fay 1982: ‘Pandis et alv/'19827 "Koch et alte 1975)" These also have 
implications for analysis of variance where some of the earliest models were 
started, but not followed (Kempthorne and Wilk, 1955; Tukey and Cornfield). 

ns As for the future I am hopeful about contributions from theory to 
applications but for two exceptions. First, mathematical statistics has not 
and will not give us complete distribution theories that will be useful 
directly, because there are too many parameters in the double complexity of 
analytical statistics from complex surveys. Second, model builders cannot 
make those complexities vanish. They will however quide us toward better and 
more comprehensive inference. Also toward better utilization and presentation 


of analytical statistics from complex surveys. 
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AN INTRODUCTION TO LINEAR MODELS AND GENERALIZED LINEAR MODELS: 
CONCEPTS AND METHODS 


Navid A. Binder! 


Univariate statistical models, linear regression models’ and 
generalized linear models are briefly reviewed. Examples of a 
two-way analysis of variance, a three-way analysis of variance and 
logistic regression for a three way layout are given. 


1. INTRODUCTION 


The purpose of this presentation is to give a bird's-eye view of some of 
the concepts used in statistical applications for modelling data 

The use of data sampled from a population to estimate means and proportions 
is now a common practice. In Section 2 we briefly review this concept and 
describe the interval estimates obtained from constructinag confidence 
intervals. 

Linear regression and analysis of variance models are often used to reduce 
multi-dimensional data to a model consisting of a few parameters. This tool 
is a valuable device for the analyst lookina for a deeper understandina of a 
complex data set. These methods are reviewed in Section 3. 

The concepts of linear reqression methods can be extended to a much wider 
class of models through the generalized linear models described by Nelder and 
Wedderburn (1972). This is particularly useful when the dependent variable is 
categorical as opposed to continuous. In Section 4 we review the structure of 


these models. 


Brief mention of appropriate diaanostics to guard against model failure and 


to detect multicollinearities is given in Section 5. 


l : : : : 
David A. Binder, Institutional and Agriculture Survey Methods Division, 
Statistics Canada. 
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2. UNIVARIATE MODELS 


2.1 Binomial Models 


Suppose we have a large population from which we will select a sample and 
we take an observation from each selected unit. If the sample size is n, we 
denote the observations by Yj, Yo, «es, ae The purpose of collecting this 
data is that we would like to make some inferences about the population based 
on this sample. For example, our population could be residents of Canada and 


our data are defined as 


nee { 1 if the person was born in Canada 
j ° 0 if the person was born outside of Canada, 


for the j-th individual selected. Based on this sample we would like to make 
some inferences on the proportion of people in the population who were born in 
Canada. 

If a simple random sample of n = 5000 residents is selected and the actual 
proportion of persons born in Canada is p = 0.85, then the number of persons 
in our sample who are born in Canada will be a random variable with a binomial 


distribution given by 
Pye 85)" (Sy 7 Peyee- 0, V2 sa cate DG: 


In this case, since we know p = .85, we can completely describe the proper- 
ties of 7¥) 12 05 ee the total in our sample who are born in Canada. For most 
statistical applications, though, we do not know all the characteristics of 
the population and we use our sample to make inferences about this popula- 
tion. For example, suppose we do not know the value of p in the previous 
example. Then we can say that the number of persons in our sample who were 
born in Canada will be a binomial random variable having a distribution qiven 


by 


AOS re ee a es y =.05 Wau. De0Us 


mn ah eS 
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Now, the usual estimator Miss p, based on this data is p = YY j/ 2000. 
We let s(p) = {p(1-p)/(5000) }?. This is our estimate of the standard error of 
D. Now, it turns out that D 15.96 s(p) is a random interval which has a 95% 
chance of including the true unknown value of p. This interval is called a 
95% confidence interval. By changing the value of 1.96 we would either 
shorten or lengthen the confidence interval, thus changing the coefficient 
from 95% to some other value. These coefficients can be obtained from 
probabilities associated with the standard normal distribution. 

We have described the binomial model via a simple random sample from a 
large population. Thus, all our inferences pertain to that population. How- 
ever, in many contexts we would like our inferences to relate to other popula- 
tions which we believe have been qenerated under similar conditions. For 
example, the number of deaths in Canada from a particular aqe-sex qroup in a 
given year may be thought of as a single realization from a binomial model, 
where each individual has the same probability of dyinq and the individual 
deaths are essentially independent. If this probability of dying is constant 
Over a number of years then the number of deaths in one year can be used to 
make inferences for other years, even though the populations are different. 
(Life insurance companies and their actuaries rely on these types of assump- 
tions in their calculations.) Providing that individual deaths are indepen- 
dent, assumptions about constancy of the probability of death are testable 
using these binomial models. 

It should be pointed out that by using some generalized linear models to be 
described in Section 4, it may be possible to improve on the assumption of 
constant probabilities for all individuals, by allowing the probabilities to 
depend on other factors such as age, sex, health status, smoking habits, 


weight, etc. 


2.2 Normal Models 


An important distribution used in modeling data is the normal distribution 


Given by 


Fe (lee cepts. 0 alye ah 21s: —, wayne. 
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The population mean is uw and is usually the parameter of interest. The 
population variance is on 

If we observe data Marea Y ort! . 22, Me from this population, our usual estima- 
tor for uw is u= Y=£2Y./n. Our estimator for the standard error of u is 


“ 2 
given by s(u) = s/n’, where 
2 2 
Ss = PaO VA Ca weer De 


As in the case of the binomial model, for large samples the 95% confidence 
interval is given by u = 1.968(u). This is a random interval which has a 95% 
chance of including the true value of u. For small samples (e.g. n < 6), the 
value 1.96 may be replaced by the appropriate value from the t distribution 
for more accurate intervals. Other confidence coefficients may also be 
obtained by changing the value 1.96 to the appropriate percentile from the 
standard normal or t distribution. 

In some applications, the assumption of constant variance is unrealistic, 


particularly in the linear models to be discussed in Section 3. A simple 


: : , : . : 2 
extension of this model is to assume that the variance of Xs is qiven by o; 
where oF = o7/w.. Here we assume that Wy, Way «eee, We are known’ weights. 


In this case wt = 5 WV fe Wi, a weighted average of the data. Also s(u) = 
1 ae 
AO» Wad where 


sg? = £ wi(Y, = i hes ye 


Confidence intervals for u are obtained analogously. It should be pointed 
out here that the weights, w), --., W, are based on the normal model specifi- 
cation and are usually unrelated to sampling weights which are derived from 
complex survey designs from finite populations. When fitting models to finite 
populations based on data from a complex survey design, the analyst may wish 
to incorporate both the model weights as well as the sampling weights in the 


estimation. 


a, Ges 


2.3 Exponential Family Models 


The binomial and normal models just described can be viewed as special 
cases of a much wider class of models known as the exponential family. The 


general form which we will use for this model is given by: 


Fly) = expLk ty ,6 = (Ce) ely; K,)I, 


where Y; takes values which do not depend on 6. 
We assume K; = cs WNETE Wl» coos w, are known. In many cases k will also 


be known. 


Example 1 (Binomial Proportion) 


We let y; = yin be the sample proportion from a binomial model based on 


n. observations. Therefore we have: 


ue Glave) 
)pd gp eer) J J : ye a0, it og Siarec oul lis 


E(y.) = p, Var(y ;) = p(1'= p)/n. 
8 = logip/(1 - p)]. 


So SS dalisn 
J J 


Bede lod + ee 


Example 2 (Normal) 


Suppose yj is normally distributed with mean u and variance o* We have: 


Xu 


aay 
— 
<< 
eee, 
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! 


ane j 
= (210 *) 2 exp {- 7 } erie Ke hyai<tereo 


EGY) alt, Var (y;) = 6 
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b(0) = 2/2. 


Example 3 (Poisson Mean) 


Suppose y; is Poisson with mean ee Letting Y ; = y /n ss we have: 
; Bir ha 


e erie eye Rey 
fly.) =e ~ (nr)! S7(m ys)! ee ee : 
J J a ee J Tie tik 
J J 
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Example 4 (x7) 
Suppose Yj has a ox, /¥, distribution. This is common for analysis of 
é ; eens 

variance and variance components models, where y, is the mean-square. Then, 


we have: 


Cy, Way eo ee re 


fly;) =, Sail exp {- y,v./(207) /1(v,/2)5 y; 2 
Elys) = 0, Var (y ;) - 20°/v., 

Q@= - 1/0%, 

Sie ai 
bC8) == log(- 6). 


As we can see from these examples, the exponential family includes a wide 


variety of common distributions. In general, we have 


= 46-3 
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where b'(*) and b"(*) denote the first and second derivatives of b(¢). 


ify ys ees y, are independent, then the maximum likelihood estimate of 8® 


is given by the solution to: 


where u = b'(8). This implies that there is a large family of models where a 
weighted sample mean provides an efficient estimator of the population mean. 


The estimated variance of u is given by 


Vn) 


2 2 
(z KV )/(E K;) 
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For large samples, the 95% confidence interval for uw is given by u +t 1.96 x 
WG eeerovidinaethemnadeleia: true: 

In cases where Kj = abe is known only up to the constant of proportionality 
K, (e.g. normal model), it will be necessary to estimate the value of Kk. The 


maximum likelihood estimate is given by the solution to: 


[ (6) eee a ] 
%, WW, ~8) & jo & : 
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Alternatively, an unbiased estimator for Vn) which is less model-dependent 


is given by 


This may be used instead to create the confidence intervals for he The 


— ioe = 


main assumption required for the validity of this approach is’ that 
Var(y.) © 1/w.. 
" J 


3. LINEAR MODELS 


3.1 One Way Analysis of Variance 


A simple extension of the univariate normal models, described in Section 
2.2, is the one-way analysis of variance (ANOVA) model. Here, in addition to 
observing one characteristic from each individual sampled, we also have a 
sub-population identifier. Some such identifiers could be aqge-sex aqroups, 


industry/occupation groups, etc. Here the model could be written as 


where the u's are population means, which differ anonq subpopulations and the 
e's are assumed to be independent normal with variances oF; z o/W, , where 
the 55°83 are known weights. In most applications the weights are constant. 


The usual estimator for HW; in this model is 


Under the model assumptions, the estimated means are independent normal 
with E(u, ) = uy and Var(u, ) = 1b W. ij’ From this, confidence intervals for 
the See means may = eee 


An alternative but equivalent description of this model is 


where ZZ w..a. = 0. Here we have 
ai ine 


ie Fr Mig Naat Wis 
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An extension of this representation is particularly useful for two-way 
and higher order analysis of variance models, to be discussed in Sections 
Bot elital 35 Be One of the main questions of interest for these models is 


whether all the means are equal. This is equivalent to wy, = U2 = «.-- = uy or 


a) = 02 = «4. = a, = 0. Standard ANOVA statistical packages (e.q. SAS, SPSS, 
etc.) are available to test these hypotheses. A related problem is: Which 
subpopulation means are equal, given that we have concluded already that not 
all means are equal? When we have no further structure (such as in a two-way 
ANOVA), this is known as the multiple comparison problems. Special treatments 


for this problem are available in many statistical packages. 


3.2 Two-Way Analysis of Variance 


The data of Table 1 has been taken from the 1975 Sri Lanka Fertility Survey 
(see Little, 1982). The cell means describe the average number of children 
ever born cross-classified by Marital Duration and Level of Education. 

The row and column means seem to indicate that the average number of 
children increases with longer marriage durations and decreases with more 
schooling. Now, the two-way analysis of variance model may be written as 


y es CaS oe SNe De eee Sey 
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where the e's are assumed to be independent normal with variances 


7 2 ‘ 
85 ak = 6 IMs se The w's are known weights. In most applications the weiqhts 
are constant. In order to estimate the parameters of this model, it is 


necessary to impose constraints on these parameters, otherwise they are not 


unique. The usual side conditions are: 


i ; : eigen =40, 

ey ki 
prt a 
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The estimators are defined by the equations: 


A A 


where 91, 92, ... correspondent to the parameter estimates Hy a, etc. The 
a's are B's are referred to as main effects and the y's are the two-way inter- 


actions. This results in the following estimators: 


= Ye lne? 
a, ieee aa ari me 8\/E Ballin 
8, ai Vaid - e ‘ 7 : Mi inei/? ; "i 5k? 
in high, i Has 5 a, ra Bs, 
where Yagy? Yarn etc. are the appropriate weighted averaaes. 
Now, the additive model specifies that Wij = H+ a, + Bs. We have plotted 


the cell means from Table 1 in Fiqure 1. The additive model would specify 
that all the lines are parallel. If the data of Table 1 are fitted to the 
additive model, we obtain the adjusted mean values in Table 2. These are 
plotted in Figure 2. As we can see, the effect of the level of education has 
been dramatically reduced after fitting this model. This is because the more 
educated women were not married for as long, so that the years since first 
marriage proves to be the important factor. However, as the analysis of 
variance in Table 3 shows, all the main effects and the interactions are 
significant. Hence the additive model is rejected. However, only 0.4% of the 
total variation is explained by the Education-Marital Durations interactions, 
whereas 49.7% of the variation is explained by the additive model. We may 
surmise from this that the additive model has led to a better understandina 


of the data and that the Education effect is not as dramatic as it first 


ao eee 
seemed. 


3.3 Reqression Formulation 


The above analysis of variance models can be considered as special cases of 


the multiple linear reaqression model, given by 


y 4 = Box 0j + BiX1j + vee + Be + Be 


where Xgj, Xijo sees we are known constants and Bo, Bly «oy B. are unknown 
coefficients. We assume that the e's are independent normal with variances 
a = o/w where the wis are known weiahts. For example, in the one way 
analysis of variance, we could let 


Xoj = 1 Loreal ies) 


>< 
I 


Fai 1 if the j-th individual is in the i-th sub-population 


Sige a, /ay if the j-th individual is in the I-th sub-population 


0 otherwise, 


ford = 15"...5 0 1, where a; is the sum of the weights for individuals in 


the i-th sub-population. In this case we have 


Hs = Bo + Bs COD mei in cs earl i cm arlis 
uy = Bo - (aq Baa a ar _4 8y_4)/ay- 
Therefore u = 8g and a; = Bs TODMeL esis \ieteceg) ole caierl ac 


A similar regression formulation is possible for two-way and higher order 


layouts as well. 
Now, for the qgeneral regression model, the estimator for Bog, ..., B&B. is 


GiVveO bY. Bip ,, “ss B.,» the solution to 
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where yj BoXo ; + Bore “ + ear 


In order to test hypotheses, perform model-building and develop confidence 
intervals for the 8's, we need the covariance matrix of the B's, This is 


given by 


Var(B) = o*A-! 


where A is the matrix with (k,2)-th entry being ? WX 3 . To estimate o%, 
we use of = e Wass - y )7/(n -r-1). , ; 

Many statistical packages routinely perform various hypothesis tests on 8 
using the estimated covariance matrix o2A-} and the critical values from the 
appropriate F-distribution (e.q. PROC REG, PROC ANOVA and PROC GLM in SAS). 

For example, Koch, Gillings and Stokes (1980) give the data in Table 4 for 
the number of physician visits per person per year in 1973 in the U.S. cross- 
classified by size of city (SMSA = Standard Metropolitan Statistical Area vs. 
Non-SMSA), Income (3 groups) and Fducation (3 groups). This data is based on 
the 1973 Health Interview Survey, a survey using a complex probability 
sample. The data are illustrated in Figure 3. 

By using a regression model and performing a number of statistical tests, 


the following reduced model was obtained: 


E(Y) Bo + BiX13 + BaXaj, 
where X1j = 1 if the j-th person is in an SMSA 


= 0 otherwise, 


1 if the j-th person has less than $5000 family income or more 


X9 j 


than 12 years education for the family head 


0 otherwise. 


The estimated parameters were 8g = 4.18 (standard error of O11), By 2.9.63 


(standard error of 0.11) and Bo = 1.12 (standard error of 0.09). The standard 


errors derived here were not those described above since the authors used the 
18x18 estimated covariance matrix from the survey to obtain the standard 
errors. This approach removes the assumption of independent error terms in 


=F <= 


the model-fitting and is a common approach for analysina data from complex 
surveys. 

In Table 5 we summarize the results. These are illustrated in Figure 4. 
We see that the model fit is quite good. We have reduced the data from 18 
values to 3 summary statistics and also have smaller standard errors (hence 


higher precision) of the estimated values. 


4. GENERALIZED LINEAR MODELS 


4.1 Regression with a Dichotomous Dependent Variable 


One of the difficulties often encountered with the linear models discussed 
in Section 3 is that the error terms were assumed to be normally distributed. 
It is true that analyses similar to those in Section 3 may be performed with 
non-normal errors, providina the variances of the errors still satisfy 


e = o/w. and the errors are uncorrelated. In this case the estimators we 


ee described yield the minimum variance linear unbiased estimates of the 
model parameters, however better estimators (i.e. non-linear estimators) may 
be available. These considerations have led to generalized linear models (see 
Nelder and Wedderburn, 1972) and robust estimators (see Huber, 1973). We 
concentrate here on the generalized linear models. 

For example, suppose the dependent variable, Yas can take on only two va- 
lues, or 1. We now want to model P = Pr(y, =.1) as Ja (functionwior = the 
linear expression X95 80 + eal + see + Xn Pre 
approaches for this problem. One is to let 9, ..., B. be the usual estimate 


There are three popular 


A 


from a stardard regression model. This is analogous to discriminant analysis 


where the variables XOis sen are not considered fixed known constants, 


rj 
but are themselves random variables (multivariate normal with constant covari- 


ance matrix) whose mean depends on the value of Y.. The problem with this 


rj Pr cannot be used directly to predict 


the value of Ps Also, in many applications the a are categorical, (e.q. 


approach is that Re = Xo; Bo tess FX 


province, occupation, etc.), thus violating the assumption of multivariate 


normality. 


Two other popular approaches are known as. probit analysis and logistic 


ie 


regression. In probit analysis it is assumed that p. = 6(z X54 Bi)» where ® is 
the cumulative distribution function of a standard normal Fandom variable. In 


logistic reoression, it is assumed that 
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Both these approaches are valuable analytic tools, and are available in 


many statistical packages (e.q. SAS, BMNP). The two approaches may be viewed 
together by letting 


For probit analysis we have uF = (ps whereas for logistic reqression 


ad 5 


we have Ne log [p3/(1 - pi]. The maximum likelihood estimate for Bo, 


B. is the solution to 
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the probit analysis we have 


where q(p.) =a? me These equations often must be solved iteratively. For 
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where $(*) is the standard normal density function. For the logistic 


regression, 
qi! p. = p. 1 ES p.)] 
; ( 3? [ j! J 


so that the parameter estimate is given by the solution to 


= BP = 


: . , a : ; : 
The covariance matrix of 8o, ..., B. is A where A is a matrix with 


(k, &)-th entry given by 


Ne Xe ae 
A = 2 kJ %J 
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This can be used to construct confidence intervals and perform hypothesis 
tests and model-building. 


For logistic regression, the covariance simplifies to 


ee ey -p.: pXaX aj 


As an example of the utility of these models, we consider an unpublished 
analysis performed by Dolson and Morin on the Canadian Health and Nisability 
Survey. The dependent variable was whether or not a person would be screened 
in as potentially disabled using the Screening Test 2 of the January 1983 
Labour Force Supplement on Disability. For details, see Dolson and Morin 
(1983). Analysis was restricted to males aged 15-64. Of the 13,897 respon- 
dents, 14.4% (unweighted) were screened in. The screened-in rates are cross- 
classified by age-qroupings, labour force participation and a proxy/non-proxy 
variable (with 3 levels: non-proxy, proxy by male or proxy by female) in 
Table 6. (The fitted values from the model to be discussed below are also 
shown.) The data are illustrated in Figure 5. 

The fitted model reduced the number of parameters from 3N to 11. The final 


model was given by 


loglp; 3. /(1 - Dail = =u + O, + B Saf ee OF i? 
where 2% a, = M5 B, =) Y= 2 ore UR es Si = 9, for the i-th age qroup, 


j-th labour force status and k-th proxy status (2 levels: non-proxy vs. 


proxy). The following were the estimated parameters. 


95%. 


Parameter Subscript Fstimate 
u -1.43 
a Aqe 15-24 -1.12 
Age 25-34 -N.571 
Age 35-44 N,N143 
Aqe 45-54 0.629 
Aqe 55-64 1.95 
B In Labour Force BF 
Not in Labour Force 0.576 
iy. Non-proxy 0.0859 
Proxy -0.9859 
§ Age 15-24, in L.F. N, 385 
Age. 25—=54, In 1. Fs 0.0938 
Age 35-44, in L.F. -.175 
Aqe 45-54, in L.F. -0.243 
Aqe 55-64, in L.F. -0.9612 
Age 12-24, not in L.F. -0. 385 
Aqe 25-34, not in L.F. -0.938 
Age 35-44, not in L.F. Uy Be 
Age 45-54, not in L.F 0.243 
Age 55-64, not in L.F 0.0612 


The fitted values are illustrated in Figure 4. 

We see that even after adjusting for age and labour force status, there is 
a proxy effect on the screening rates. This proxy effect does not seem to 
depend on the sex of the proxy respondent. Also, there is no interaction 
between the proxy and the aqe/labour force status variables. This model does 
not necessarily imply a proxy bias, but it indicates that a proxy bias may 
potentially be present. Without a special study such as a re-interview pro- 
gram for the proxy respondent, it is impossible to definitively conclude the 


existence of a proxy bias. 


4.2 Generalized Linear Models 


In the previous section we discussed a large class of linear models related 
to the binomial model, of which probit analysis and logistic rearession were 
special cases. We now extend these to the exponential family as proposed by 
Nelder and Wedderburn (1972). 


As in Section 2.3, we assume Y 5 has probability function given by 


= Fhe 
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Wesleteni = qu) =P a Sails be the linear component of the model, where 
ah 


g(*) is a known function. 


Now the maximum likelihood estimates of B are qiven by the solution to 


OF. - u.)X 
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Nelder and Wedderburn (1972) have shown that a reasonable method for 
estimating 8 is given by performing a number of weighted least-squares 
regressions, updating the weights and the dependent variables on successive 
iterations. This is called iteratively re-weighted least squares. In 


particular, the weights for the t-th iteration are given by 


A 1 
com 


ee ate 
yee fan ot 
J J 
and the dependent variables on the t-th iteration are qiven by 
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fies (tan) =the aterat tonnor B is then the solution to 
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The estimated covariance matrix of B is given by hae where the (k, &)-th 


entry for A is 


=e 


Aa 
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This implies that many standard weighted least-squares packages could be 
invoked to perform analysis of these aeneralized linear models. 
For example, a common analysis of contingency tables, called log-linear 


models assumes a basic Poisson model with loa ne = ~ X..8.. Here we have 
: eee 


Vie ie 
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q(u.) = log u., 


so that the iteratively reweigqhted solution is given by assigning 


Gp aon 
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Hence, models similar to those described in Section 3 can be analyzed 


analogously using the generalized linear model formulation. 


5. DIAGNOSTICS 

Linear reoression methods have been known now for over a century; see 
Hocking (1983) for a review of developments over the last 25 years. In more 
recent years attention has been focused on difficulties encountered when there 
is multicollinearity in the variables (leading to large variances of the para- 
meter estimates) and when the models may fail. Some of these diagnostics are 
now available in SAS and SPSS-X. 

The methods discussed in this paper extend linear rearession to a much 


wider class of problems. Newer diagnostic techniques for models of this sort 


= 96" = 


are discussed in Landwehr, Preqibon and Shoemaker (1984). 


In many statistical applications, the proposed model is only used as an 


approximation to reality. Therefore, the user of these models should employ 


these diagnostic tools in the course of the analysis. 
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Table 1: Mean Number of Children Ever Born, by Marital Duration 
and Education Level. Sri Lanka 1975 (from Little, 1982) 


Level of Education 


Years since 
First Marriage 


25+ 


Column 


Table 2: 
Years 
Since 
First 
Marriage 
Mo Raw Mean 
Adjusted Mean 
Interaction 
» SY Raw Mean 
Adjusted Mean 
Interaction 
AG) 4a Raw Mean 
Adjusted Mean 
Interaction 
>a Raw Mean 
Adjusted Mean 
Interaction 
20) — 24 Raw Mean 
Adjusted Mean 
Interaction 
25+ Raw Mean 
Adjusted Mean 
Interaction 
Column 


Interactions for Mean Number of Children from Table 1 


No 
School 


0.96 
Ee)! 


Level of Education 


ae PO 


Table 3: Analysis of Variance of Data from Table 1 


Sum of {Proportion 
Squares 


Source 


Main Effects 


Marital Duration 27402.684 0.493 1340.990 
Education/Duration 225.6535 0.004 18.395 
Interactions 

DurationxEducation 206.965 0.004 33376 
Residual 0.499 


27729 .848 


Total 55565 .031 


Table 4: Physician Visits per Person per Year by Residence Size. 


Family Income and Education of Family Head, U.S. 1973 


Education Family Income 


in oo eee Seis EE 
Years 0 - 4999 | 5000 - 14999 15000 or more 


SMSA 

[Ses than tie 6.15 4.73 4.82 
(0.18) (0.13) (0.25) 
12 6.17 4.98 4.70 
(0.41) (0.17) (0.18) 
Mora tthan (2 6.31 6.08 5.66 
(0.49) (0.19) (0.16) 

Pe ae ~ SF" RSA 


lesssthan ce 5.08 4.14 4.42 
(0.26) tO, 159 (0.37) 

12 5.36 4.32 4.49 
(N.44) (9.19) (0.33) 

More than 12 4.58 5.06 4.48 
(0.58) (0.29) (0.31) 


Note: Bracketed figures indicate standard errors of estimate. 
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Table 5° Estimated Physician Visits from Table 4, 
Original and Fitted Values 


Family Income 


5000 - 14999 | 15000 or 


SMA 


Education 
(in Years) 


Less than 12 Original 6.15 (0.18) 
Fitted 5.95 1 (0407 ) 
Difference 


12 6.17) (0847) 


5.95 (0.07) 


Original 
fitted 
Difference 


More than 12 | Original 6.51 (0749) 
__ Fitted 5955 OF.07) 
Difference 


Less than 12 Original 
Fitted 
Difference 
12 Oriqinal 
Fitted 
Difference 


More than 12 Original 
Fitted 


Difference 


Pe 


Table 6- Unadjusted and Fitted Screened-in Rates from Test 2. 
Canadian Health and Disability Survey, Males Aged 15-64, 
by Labour Force Participation and Proxy Status, Canada 
January 1983 (Unweighted) 


Pe | Newt Non-Pre -Proxy | Male Proxy co Pasaie Proxy 
In Labour or a OOM a 
15 - 24 | Unadjusted .065(.0067) .055(.0143) "056 (.0069) 
meluced .065( .0051) -056(.0044) .056(.0044) 
Difference nao -.001 . 000 
- 34 Unadjusted .085(.0058) .058(.0252) -169(.NN69) 
Fitted -085( .0048 ) -071(.1046) .071(.046) 
Difference .onn motel 15 6. my 13 
- 44 Unadjusted 113(.0079) .029(.9290) -194(.0N86 ) 
itted | -111(.0064) .093(.0059) -093(.9059) 
Difference N02 -.064 01 
= 54 Unadjusted 180(.N109) N82(.0351) <158( 1120} 
meh aceed 177(.0088) -153(.9N83) «153( OAS) 
Difference N03 -.071 .001 
- 64 Unadjusted .284(.0150) 207 (BIS?) .250(.0183) 
Fitted .283( .0124) +249(.0124) .249( 0124) 
Difference 001 -.042 .001 
Force 
Unadjusted <U27) .071(.0190) D740 noe4) 
Fitted .0078) .079(.0065) .079(.N065) 
Difference -.008 -.005 
Unadjusted 0239) 367(.1450) .227( 0365) | 
Fitted _ 192( 0213) 167.0194) .167(.0194) 
Difference 200 - 060 | 
Unadjusted .9372) 6855031501) .324(.0544) 
Fitted .0309) .320( 0299) .320(.0299) 
Difference 135 004 
Unadjusted .0361) 625(.1712) .454(.9505) 
Fitted . 0293) .483( .0301) .483(.0301) 
Difference «12 929 
Unadjusted .0220) .563(.124N) .591(.042N) 
Fitted .0194) .543( 9217) .543(.9217) 
Difference .020 048 


NOTE: Bracketed figures are Standard Errors 


=: a 


YEARS SINCE 

i FIRST MARRIAGE 

—— Ls item 

ak i Seg 
A ee ——_-———. 25+ YRS 
5 ee ore ee 
edie ee oes 

(Gant, bad... SEMA Ae TF he fASENONYRS 
34 10-14 YRS 

Mm Tce: ee ues co ane ae 5— 9 YRS 
2 
1 nee foe ) ee oe ee O- 4 YRS 
O- 


NONE 19 6—9 10+ 

YEARS OF SCHOOLING 
Figure 1: Observed Means from Sri Lanka Fertility Survey, DTD. 
Data source: Little (1982). 
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Data source: Little (1982) 
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Figure 3° Observed Mean Number of Physician Visits per Person 
per Year. U.S.A., 1973. 
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ADJUSTING SUB-ANNUAL SERIES TO YEARLY BENCHMARKS 


Pierre A. Cholette! 


This paper proposes a modification to the method of Denton (1971) 


for adjusting sub-annual series to yearly totals. These totals 
originate from more reliable sources _ and constitute annual 
benchmarks. The benchmarked series derived according to the 


modified method is more parallel to the unbenchmarked series than 
this is the case with the original method. An additive and a 
proportional variant of the method are presented. These can easily 
be adapted for flow, stock and index series. Also presented are a 
few recommendations about the preliminary benchmarking of current 
data and the management of "historical" estimates of the series. 


1. INTRODUCTION 


In many cases, the statistician obtains sub-annual data of a series from 
one source of data (such as a sample survey): and, the corresponding annual 
benchmark values from another more reliable source of data (such as a 
census). The annual sums of the observed sub-annual values are generally not 
equal to the annual benchmark values. Such sub-annual series require adjust- 
ment to annual benchmarks, that is benchmarking. 

The solution proposed by Denton (1971) (and generalized by Fernandez in 
1981) consists of finding a sub-annual series which would display the movement 
of the available sub-annual series as much as possible and whose annual sums 
(or averages) would match the more reliable annual benchmarks. The level of 
the resultina series would then be given by the annual benchmarks, whereas its 
movement would be dictated by the original sub-annual series. In other words, 
the adjusted or benchmarked series should run as parallel as possible to the 
original, while still satisfying the annual benchmarks. This paper suqgests a 
modification to Denton's specification which makes the original and the ad- 


justed series even more parallel. 
We follow the model of Ehrenberg (1982) for the presentation of scientific 
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papers. The reader will be exposed to the illustrations and results first; 


and the methodoloqical details, afterwards. 


2. ILLUSTRATION OF THE RESULTS 


Fiqure 1 shows the corrections (x, - Z,) made to the oriqinal series zy 
according to the additive solution (with first differences) of Denton and 
according to the corresponding solution proposed in this paper. Since the 
corrections are to be added to the oriqinal sub-annual series Zh the adjusted 
series Xt will be completely parallel to the original series, if and only if 
the corrections are constant. In the figure, this happens only for the cor- 
rections derived under the method proposed in this paper. 

Figure 1 presented a trivial and ideal case which allowed the solution of 
constant corrections: All the average annual discrepancies, the differences 
between the annual benchmarks and the annual totals of the original series 
(divided by the number of months per year), were constant. Figure 2 displays 
a more realistic case, where the five average annual discrepancies vary about 
200. As in the first example, the corrections derived by the herein proposed 
method are much more constant, especially in the first year. 

As explained below, Denton's method does not only minimizes the change in 
the corrections (to make them as constant as possible) but also the size of 
the first correction. This can be seen both in Figures 1 and 2, where the 
first corrections are close to zero. The alternative solution, on the other 
hand, only minimizes the change in the corrections. Graphically this consists 
of fitting a curve through the average annual discrepancies, which is as flat 
as possible and which spans the same annual surfaces as the average annual 


discrepancies. 


3. KEEPING THE ORIGINAL AND THE BENCHMARKED SERIES PARALLEL 


Resuming the additive first difference formulation of Denton as well as his 


notation, the desired series Xp minimizes the following objective function 


(x) =f , 2 
p(x) = 2 = 42, CAO - 240)" x0 = Zo, (1) 


i) = 


where Zo stands for the original sub-annual series at time t. This function 


is minimized subject to the equality constraints between the annual sums of 


the values obtained and the available benchmarks y;? 


ik 
eres Xe = Va» A FS ln 32 gh ov oteny: ihe (2) 
where k is the number of "months" per year. 

Denton justifies hypothesis xg = zg claiming that it is legitimate to 
assume the equality of the last fitted and observed values prior to the 
estimation interval. Objective function (1) would then mean that the adjusted 
series xX, should have the same slope as the original series Z43 and therefore, 
that the slope of the differences between the two series should be minimized 
(subject to the constraints). However, after substituting xg = zo, objective 
function (1) can be rewritten as: 


Be --71)° + 2 (a(x, - 2,))%. (3) 


This transformation emphasizes that the assumption xg = Zg implies minimizing 
the size of the first correction. As illustrated in Fiqures 1 and 2, minimiz- 
ing the first correction pulls the correction curve towards zero at the start 
of the series. This produces a wave in the first year which is transmitted to 
the other years. This wave in the corrections prevents, by definition, the 
maximum parallelism between the observed and adjusted series. 

The specification proposed here simply refrains from postulating xg = Zo 


and yields the following objective function 


n 5 ae 
Bex) a= ee CACC Ea 4) 


subject to the same constraints of equation (2). 


In linear algebra, the constrained objective function is written 


Wx; ges (& - 2)"A (x - 2) - Dally ~ Bix)s 


a ¥36 = 


where the vectors and matrices involved are: 
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Vector gq contains the Lagrangian multipliers. 
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(6) 


(7) 


(8) 


Variables n (= mk), m and k 


respectively stand for the number of observations and of years in the series 


and the number of months per year. 


The normal equations associated 


du/dx = (A + A')(x - Zz) +2 


du/dqg = 2(B'x - y) = 9 


and yield solution 


x Le ks 
is) EO ei ep 
Substituting identity y = B'z 


cies, gives 


with objective function (5) are 


eel! 
(n+m) x (n+m) 
+ r, where r contains the m annual 


(9) 


(10) 


discrepan- 
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This reformulation of the solution reduces computing time in the applica- 
tion of the calculated weights compared to formulation (10). Also note that 
once the weights ny are obtained, they can be used for any number of series 
having the same number of observations. Furthermore, we recommend (Cholette, 
197/2,-secc10n 6; 1979), 4.3)° to comptite W for a 5-year interval and to use it 
in a moving average manner (moving one year at the time) for series of 5 years 
and more. Apart from saving on calculations, this procedure qenerates only 
two revisions in the estimates (ceteris paribus) when new years of observa- 
tions are added to the series. 

Denton solves the inversion in equation (10) by parts. This is impossible 
here since matrix A is singular. The overall matrix however is not singular 
and can be inverted. 

In fact, the method developped herein uses the solution proposed by Root, 
Feibes and Lisman (1967) to interpolate between annual data in the absence of 
sub-annual information. Solution (11) exactly consists in interpolating 
between the annual discrepancies with the method of these authors and in 
adding the resulting estimates (the corrections) to the original sub-annual 


series. 


4. PROPORTIONAL VARIANT 


The proportional method now presented in this section is also a variant of 
Denton's proportional method, from which xg = zo was removed. As in Section 
2, the objective function still minimizes the sum of the squared differences 
between the slopes of the original and desired sub-annual series (Z, and xy). 
Each term in the sum is weighted however by the value of the corresponding 


sub-annual observation: 


n n 2 | 
p(x) = aa (A(x, - zeae) * = 2, (A(x, /24))*- 12) 


BAe =: 


This variant is suitable for series with stronq seasonality, when it is 
thought that seasonal trough months cannot account for the annual discrepancy 
as much as seasonal peak months: The size of the corrections are proportional 
to the level of each observation, as illustrated in Fiqure 3. The low obser- 
vations get smaller corrections than the seasonally higher observations, 
although the minimized proportional corrections x, /Z 4 are as flat as permitted 
by the annual discrepancies. Note that with the proportional variant all 
observations must be positive and that all the adjusted values will also be 
positive. 

It can also be shown (Cholette, 1978, Section 3; 1979, 3) that the pro- 
portional variant is a linear approximation of the strongly non-linear qrowth 
rate preservation method (Smith, 1977: Helfand et al., 1978), which would have 


the following objective function: 


D Oxides . (x,/X,_, - ay [ze GOES (13) 
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The approximation is exact in situations of constant annual proportional 
discrepancies on the estimation interval. 
In linear algebra, the constrained objective function associated to the 


proportional method is 


OG) eee) Pa Len (X 2) eer) (yes BO (14) 


where Z-! is a diagonal matrix with elements 1/z,, 1/z2, ... The solution has 


the same structure as the additive variant (Z-!A Z-! replacina A in (11)) and 


writes: 
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Unlike the weights in the additive variant however, weights We of the pro- 


portional solution must be computed for each series and even for each 


eT 


application interval of a aqiven series. 


5. STOCK AND INDEX SERIES 


The additive and proportional variants of the method presented above are 
designed for flow series, whose annual values correspond to the sum of the 
sub-annual values. The solutions can very easily be adapted for stock series, 
whose annual values are associated to only one sub-annual value (usually that 
of the last month); and for index series, whose annual values correspond to 
the average of the sub-annual values. For a quarterly stock series, for 


instance, one merely has to redefine the component vector j of matrix R as 


+4 
J = OO 4] : 
1x4 


and, for monthly index series as 


SY 2 [1/12 1/12 2... 1/121. 
1x12 


6. DISCUSSION 


6.1 Historical Data 


There is a lot of confusion regarding the interpretation of assumption x9 = 
Zq of Denton. In that respect, the author writes: "It is assumed that no 
adjustments are to be made to the original series for years outside the range 
from year 1 to m, inclusive." (p. 19M, above equation (3.2)). 

If these years are left untouched because they never had any benchmarks, 
the solution proposed by Denton is defendable: No corrections result for 
years -1 and 0; and small and gradually introduced corrections, at the start 
of year 1. (Remember that xg = Zqo implies minimizing the first correction. ) 
The resulting adjusted series is then continuous as illustrated in Fiqure 4 by 


curve ADEB. 
However, if the first years are left untouched because they were already 


ae 


benchmarked and are now considered "historical", we do not aqree with assump- 
tion xg = Zoe Indeed, this assumption will generally produce a discontinuity 
between years O and 1, as shown in Figure 4 by curve A'CDEB. Years -1 and Nn 
have already received corrections of magnitude around CD, whereas the start of 
year 1 receives corrections which are as small as possible. 

In order to "freeze" the historical data after a certain number of years, 
two solutions are possible. First, one can explicitely specify the freezing 


constraint in the objective function which becomes 


p(x) = (Ox, = 23) = (xg = 20))? + 12, (A (my = 24))?, (16) 


where (xg - zg) is known and equal to the last correction used for historical 
year 0. This correction is generally not equal to zero (Cholette, 1979b, 
1983). This specification amounts to determining the starting point of the 
correction curve. 

Second, a less specific but equally effective solution consists of applying 
the methodology already proposed in this paper (additive or proportional 
versions) as a moving average, which moves one year at the time. With a 
5-year estimation interval, for instance, the estimates automatically become 
final after two years of revision; and, after one year, in the case of a 
3-year interval (Cholette, 1978, section 6 a; 1979, 4.3). The resulting 


benchmarked series is continuous, as illustrated in Figure 4 by curve A'CR. 


6.2 Implementation 


The practioners of benchmarking have a tendency to feed to the benchmarking 
programme the already benchmarked years of data followed by one year of 
unbenchmarked data (all accompanied by their benchmarks). For methodologists, 
it is obvious that one must always submit the unbenchmarked data (with the 
yearly benchmarks). Feeding benchmarked data will generally induce an 


artificial seasonal movement in the resulting benchmarked series (Cholette, 
1978, Section 6b). 


= Le 


6.3 Preliminary Benchmarking of Current Data 


A final comment is in order. During a current (uncompleted) year, one 
cannot calculate growth rates, for instance, between the benchmarked seament 
of the series (AB) and the unbenchmarked seaqment CLOT, Doing so usually 
produces a discontinuity BC between the two seaments AR and CD as illustrated 
in Fiqure 5 by curve ARCN. 

Two solutions are then possible. One, the inter-temporal comparisons are 
based only on the unbenchmarked data. Two, the current data are preliminarily 
benchmarked by repeating the last available correction BC for the current 
year. (Note that including the incomplete current year in the objective func- 
tion (4) (or 12) would yield identical preliminarily benchmarked values.) One 
can then compare the benchmarked seqment AB with the preliminarily benchmarked 
seqment BE as illustrated in Figure 5 by curve ABE. We favour this second 


alternative. 


6.4 Relation with Other Methods 


The Denton (1971) benchmarking method, the modified Denton method (presen- 
ted in this paper), the methods of Glejser (1966), of Boot, Feibes and Lisman 
(1967), of Lisman and Sandee (1964), and of Bassie (1939) could be reffered to 
as univariate methods. No series other than that considered and its annual 
benchmarks enter the benchmarking process. On the contrary, the methods by 
Friedman (1962), by Chow and Lin (1971), by Somermeyer, Jansen and Louter 
(1976) and by Wilcox (1983) are multivariate. Auxiliary series are used in 
the computation of the desired series. 

For instance, Chow and Lin (1971) proposed a method to obtain the desired 
sub-annual series from yearly totals and from related series. The movement of 
the resulting series is as much as possible similar to the movements of the 
related series (and the series obtained satisfies the annual constraints). 
Fernandez (1981) observes that the Chow and Lin method can produce movement 
discontinuities between the years. He then proposes a synthesis of the Chow- 
Lin and of the Denton methods. The combined method elimates the inter-annual 
discontinuities, but still relies on the hypothesis xg = Zo. As illustrated 
above, this hypothesis often introduces spurious fluctuations in the calcul- 
ated series. We would think that it should be possible to refrain from the 


hypothesis in the case of Fernandez as in the case of Denton. 


men 2 
7. SUMMARY AND CONCLUSIONS 


Denton (1971) intended to keep the original and benchmarked series as 
parallel as made possible by the annual discrepancies. This paper suggested a 
modification to the benchmarking method which makes the original and bench- 
marked series more parallel than is the case with the original method. This 
improvement holds both for the additive and the proportional variants of the 
method. We suspect that the generalized multivariate method by Fernandez 
could be improved in the same direction. 

The method proposed can very easily be adapted for flow, stock as well as 
index series. 

Before making intertemporal comparisons between the benchmarked and current 
data, it is essential to preliminarily benchmark the current data (in the 
manner proposed). 

The suggested 5-year moving average implementation of the method will 


automatically "freeze" the past estimates after two years of revision. 
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Figure 1: Corrections (x, ~ z,) made to the unbenchmarked series according to 
Denton's method (dashed line) and according to the method proposed in this 


paper (solid) in an ideal situation of constant annual discrepancies. 
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Figure 2: Corrections (x, - Z,) made to the unbenchmarked series accordina to 


Denton's method (dashed line) and according to the benchmarking method 


proposed in this paper (solid) in a situation of variable average annual 


discrepancies (dotted). 
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Figure 3: Original 


series (solid curve) 


and benchmarked series (dashed) 
according to the proportional variant of the benchmarking method proposed in 


this paper (in a situation of constant annual proportional discrepancies). 
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Figure 4: Benchmarked series according to Denton's method, 


when there are no 
benchmarks for year -1 and 0 (curve ADEB) and when there are benchmarks and 


year -1 and 0 were already benchmarked (A'CDEB): and according to the method 
proposed in this paper, 


applied in a moving average manner, when there are 
benchmarks for years -1 and O (A'B). 
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Figure 5: Continuity between the benchmarked series (dashed curve) and the 
preliminarily benchmarked series (dotted) and discontinuity BC between the 


benchmarked (dashed) and the unbenchmarked (solid) series. 
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EXAMINING EXPENDITURES ON ENERGY 


Louise A. Heslop! 


Using data from the Family Expenditures Surveys over time, consumer 
expenditures on in-home and transportation energy from 1969 to 1982 
are being studied. This article briefly summarizes some of the pro- 
cedures being used to explore the data, summarize it and develop in- 
sights into shifts in consumption for policy implications purposes. 
With such a complex data set and such a complex, multi-faceted sub- 
ject for analysis some effort must be made to reduce information 
flows and at the same time increase the information content of each 
factor of both input and output in the analyses. 


1. THE ENERGY ISSUE 


To some, energy conservation may be a dead issue. There is no shortage of 
energy (maybe never was): prices for eneraqy have stabilized. 

Eneray matters dominated the 1970's having major impacts on the world eco- 
nomic order and creating international strife. Domestically they impacted 
drastically on federal - provincial relations and business - government rela- 
tions and on family budgets: caused the restructuring of the manufacturing 
base, the auto industry, etc. Despite its reported demise as an important 
issue, energy consumption and prices remain as high priority concerns of 
consumers, businesses and governments. Energy conservation has lost its 
sparkle but not its real value. 

The research I will be reporting on briefly has been developed in consulta- 
tion with policy makers in Consumer and Corporate Affairs Canada and Energy, 
Mines and Resources Canada which continue to run active research programmes on 
consumer energy use and conservation. The project structure has taken their 
interests, orientations and limitations into consideration. 

Also, within the last five years an international qroup of social scien- 
tists has begun a series of research and information exchanges on consumer 


behaviour and energy use. As a member of that group I have been keenly aware 
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of the problems and prospects and the current state of knowledge and research 


techniques of that qroup. 


2. PROBLEMS IN ENERGY RESEARCH 


Perhaps the major problem in studies of consumer energy use has been to 
obtain reasonably reliable measures of use from sufficiently large and repre- 
sentative samples. Getting such data over a period of time, especially a time 
period spanning the infamous 1973 oil embarao period, would send a researcher 
into Nirvana. The Family Expenditure data collected by the Consumer Income 
and Expenditure Division of Statistics Canada come close enough to these 
requirements to at least set one's heart fluttering. It is a series of retro- 
spective recall studies conducted for the years 1969, 1972, 1974, 1976, 1978 
and 1982. So it covers the time period of interest for a large sample and the 
sampling technique used ensures that the design is representative of Canada 
for those areas studied, usually urban centres. Additionally it contains a 
great many other variables of interest in any study of energy use, e.g., home 
ownership, some house characteristics, vehicle and appliance ownership, family 
characteristics and expenditures on other categories of consumer goods and 
services, etc. 

Most studies which attempt a measure of consumer expenditures rely on 
recall or file checking by respondents. There are obvious problems with the 
accuracy of such data on an individual basis. The problems are less restric- 
tive with very large samples. For most independent studies, the costs of such 
large samples are prohibitive. However, FAMEX sample sizes are very larqe. 

Only one major study in Canada has used independent record checking, ob- 
taining records from suppliers by household with the permission of the house- 
holder, but through this technique was able to obtain electricity use records 
on less than half of its sample. Natural gas and oil records were obtained on 
only about one-third of the sample. This procedure of record checking is 
highly accurate, removes the problems associated with recall, especially over 
long periods of time, and of reporting bias of respondents. However, practi- 
cally it is impossible to use for large samples across the country. 


Although the FAMEX Study uses recall procedures, the information on eneray 


yee 


expenditures are not likely to be as biased as in a study specifically 
designed to record energy behaviours since respondents are not sensitized to 
the subject of the study. Also the data from pre-eneray crisis periods was 
collected in the same way as that since the crisis, again reducing the 
likelihood of response bias. So the FAMEX data set offers a unique 
opportunity to examine a very larqe set of samples during a very important 
period of time. 

The data set is not without its problems, some because of the sampling 
procedure and some because of the inherent complexity of any study of eneray 
use. Changes in expenditure categories and their contents, especially those 
other than energy, have required that we manipulate the data considerably to 
create consistency across years. It is not possible to track in-home energy 
expenditures for those families who do not pay for eneraqy directly, i.e., 
apartment dwellers with central metering and roomers. Some researchers have 
imputed values to these households based on their rents but we chose not to, 
and instead have chosen to restrict our study to those households who have the 
ability to monitor and affect their own energy use. These households are the 
consumer groups who will be the focus of any qovernment programmes to alter 
consumer comsumption. 

There are several factors which make the study and the altering of energy 


consumption of households difficult: 


- Capital commitments restrict the ability of the household to respond in 
the short-term and increase the cost of response - e.qg., house size, 
number and type of appliances, size and number of vehicles. Some studies 
have noted that home characteristics alone may account for 24% of in-home 
energy consumption. Family size may be considered as a capital 
commitment as well. 

- Flow feasibilities - There are restrictions in the ability to change the 
amount and types of fuels used depending on the technology and fuels 
available under different circumstances and for varying amounts of money, 
e.g., natural gas heating is not available to rural _ residents: 
instantaneous changes can not be made in the type of home heatina fuel 


used. 


- Exogenous factors affect the amount of enerqy needed for similar 
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performance in different situations, e.a., weather, distances between 


points in cities, etc. 


3. SUMMARIZING INFORMATION INPUTS AND MAXIMIZING INFORMATION OUTPUTS 


With such a complex data set and such a complex, multi-faceted subject for 
analysis some effort must be taken to reduce information flows and at the same 
time increase the information content of each factor of both input and out- 
put. There are several ways of doing this, some of which we will be usinq, 


they include: 


a) Constructing Complex Input Variables - to reduce the number of factors 
being studied to the most salient ones. 

i) Discontinuous complex input variables were created by combining in-home 
and transportation energy consumption but not as continuous variables. Rather 
groupings were created to develop a set of typologies whose characteristics 
can then be examined for differences. In this case the qroupings were devel- 
oped by creating expenditure quartiles for each energy category, collapsing 
the two middle categories, and then combining the two resulting three cells 
into a nine cell matrix of interrelated categories (see Table 1, source: 
McDougall, Ritchie and Claxton). In particular, the corner cells are of 
interest in contrast to each other and to the middle cell. This typology was 
developed in an earlier study for Consumer and Corporate Affairs Canada. So 
comparing the output from the FAMFX data to the data set used in the CCA study 
will be of particular interest. Comparing the characteristics of these groups 
over time will also be of interest. For example, do the Churchmice continue 
to be impoverished Canadians (involuntary simplicity) or is there any indica- 
tion that there is some voluntary embracing of low eneray, lifestyles? In 
Table 2 the characteristics of three cells of the typoloay from two different 
years are compared - the Churchmice, the Roadrunners and the Hippos. Looking 
first at the Churchmice, information on a selection of possible analysis vari- 
ables is shown across two different years, 1974 and 1978. To simplify for 
this presentation only the rankings of the cell within the typology set of 


cells is given. Characteristically those consuming the least amount of eneray 
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have had the least resources in general, i.e., the lowest incomes, the lowest 
levels of education, the oldest. These characteristics are evident for the 
Churchmice in 1974, they also have the lowest levels of consumption for all 
the expenditure categories shown. Although they are the oldest group they do 
not have the lowest number of very young children. Probably this qroup con- 
sists of a mix of senior citizens and single parent households (probably 
headed by women) with young children. Note that this group also has the 
lowest number of full-time earners (F-T earners). In 1978 the general picture 
is still the same except that this group is no longer the oldest. In fact the 
oldest group is in the adjacent cell to the right in the typology (not shown 
here). It would seem that in 1978 the very old are consuming a relatively 
larger amount of in-home enerqy. Perhaps this group is financially better off 
in 1978 than in 1974 or perhaps they have been unable to hold the line on 
energy expenditures as prices have risen. 

In 1974 the Hippos also fit expectations. They seem to be middle-aged with 
large numbers of children 5-16 years of age. The "full nest" family, they 
spend the largest amount on most expenditure cateqories. They are also the 
most highly educated. In 1978 this is no longer true as the education ranking 
of this cell has dropped. Also this group no longer has the highest shelter 
expenditure. Some suggestions for these observations may be that those with 
the largest homes and the highest education have begun to modify their homes 
to reduce energy expenditures. 

The Roadrunners have changed also. In 1974 they were the youngest group 
with very small families. In 1978 they appear to be characterized as young 
families with young children. One of the most dramatic changes for this qroup 
has been that their alcoholic beverages and tobacco expenditures have dropped 
dramatically. 

The significance of these changes can be determined with appropriate sta- 
tistical tests. The purpose of this discussion was to introduce the idea of 
searching for meaningful typologies within the data. Pictures of the life- 
styles of the groups emerge which can be very useful in furthering conserva- 
tion programmes directed at each qroup. 

Further analysis may look not at level of expenditures but at percent of 
expenditures. Such an analysis will reveal the characteristics of those who 


are most heavily burdened with energy bills. 
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ii) Continuous complex input variables can be constructed to eliminate the 
effects of variables known to have very large effects, but ones which are 


difficult or impossible for consumers to manaae. 

In-home enerqy expenditures can be examined for factors related to them, 
but since one of the main determinants of in-home energy expenditures is house 
size, this size factor can be absorbed into the input variable to allow for 
examination of other more relevant (from a policy perspective) factors. So 
instead of in-home energy expenditures, in-home expenditures/room are exam- 
ined. Takina this one step further, climate and weather variances from year 
to year may be controlled for by looking at expenditures/room/deqree day. 
This last factor is added to the data set by city by year. Deqree day data 
for each year for each city were obtained from Environment Canada. Table 3 
indicates how the figures change as the factor studied becomes more complex 
again across two of the years of data. A comparison of the two years and 
differences in the measures of change between years suggests the importance of 


refining the measure to improve understanding of the process. 


b) Constructing summary output variables to examine the structure of the 
data - Example of regression coefficients. 

In Tables 4-6 some regression outputs are presented. Three models are 
examined. In each succeeding model the dependent variable becomes more com- 
plex. In so doing the factors known to impact siqnificantly on energy con- 
sumption can be controlled for and the effects of the remainina variables 
examined more constructively for any siqnificant explanatory power. 

In these analyses no attempt has been made to deal with the problem of the 
complex sampling desiqn. A future analysis will do so using the Taylor lin- 
earization procedure and results will be compared. However, the results from 
both a weighted and an unweighted sample are shown for 1974, As can be seen 
the values of the coefficients chanae very little and their significance or 
lack thereof does not change. Because of the restrictions indicated and also 
the fact that the very large sample sizes are used here produce siqnificant 
results under conditions of very slight differences, it is advised that great 


care be taken in viewing these preliminary results for purposes of this dis- 


cussion. I will only note the variables significant at the .01 level and 


beyond and then only their sign. 
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In the independent variable list dummy variables are used in the first and 
second models for city and in all three models for type of dwelling type. The 
unspecified condition is Ottawa for city and single detached house for 
dwelling type. 

In 1974 house size, some city variables, total expenditures, aae of head 
and family size and some house types are significant. Large families with 
high total expenditures living in single detached homes in St. John's consume 
the most. Western cities consume less than the east, and all other housing 
types consume less than detached houses, although duplexes not significantly 
so when number of rooms is controlled for. The unweighted results are similar 
to the weighted. 

When the dependent variable is changed to $/room and number of rooms is 
removed from the list of independent variables the qeneral pattern remains. 
However, family size is no longer significant (probably closely tied to dwell- 
ing size only), and education of family head becomes siqnificant with a neqa- 
tive sign. Those with less education consumed more, all other things being 
equal. Finally duplexes become significant with a positive siqn, so when 
number of rooms is controlled for, duplexes use more eneray than detached 
houses. 

In model 3 climatic conditions are taken into account by controlling on 
deqree days in the dependent variable and the list of cities is dropped from 
the independent variable set. 

It should be noted that the value of the coefficients drops so dramatically 
because there are between 4000 and 7009 degree days in these cities. So the 
small value of the coefficients does not mean they are unimportant. Total 
expenditures remains significant as does education of the family head and the 
rowhouse effect. An important thing to note is the drop in the value of the 
adjusted R-squared. In fact the independent variables remaining in the equa- 
tion do not do very much to help in explaining variance in the dependent vari- 
able. Other more useful variables should be sought. 

When we compare just the unweighted 1974 and 1978 results, in model 1 some 
change in the Vancouver parameter can be noted and in the importance of semi- 
detached and duplex housing over detached houses. 

In model 2 again the major change is in dwelling type effects. Finally in 


model 3 only the rowhouse variable shows any difference from the detached: 
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education of the head is again important, but in 1978 age of head is siqnifi- 
cant with a positive coefficient. Some improvement is seen in the R-squared 
for 1978, but it is still very low. 

This cross-year comparison from a policy perspective suqaests perhaps that 
improvements have been made in the quality of the detached housing stock in 
Canada. From a methodological perspective it indicates the importance of 
choosing the dependent variable with care. 

As was earlier noted, much additional analysis and re-analysis will be done 
using the reqression procedures available to refine these results and take the 
sampling design into account. 

As I noted earlier the FAMEX data sets have their limitations but they also 


contain a wealth of important information which should be fruitfully explored. 
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Table 1: Enerqy Consumption Taxonomy - Labels 


Level of In-Home Energy Consumption 


High 
222 Mills kd 


CHURCH MOUSE BEAR 
.5% of sample| 9.8% of sample 2.5% of sample 


BEAVER 


Gasoline i ” of sample| 33.7% of sample |12. 3% of sample 


Consump- 
tion 


HIPPO 


12.6% of sample 6.1% of sample 


Source: See reference list. 
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Table 2: Rank among Typology Cells 


CaN cM Ga 


Education of Head (low-hi) 1 1 9 7 7 g 


Aqe (old - ynq) 

F-T Earners (low-hi) 
Family Size (low-hi) 

Child Less than 5 (low-hi) 
Child 5-15 (low-hi ) 

Food at Stores (low-hi) 
Food at Eating Places (low-hi) 
Shelter (low-hi) 

Clothing (low-hi) 

Personal Care (low-hi) 
Medical (low-hi) 

Tobacco & Alcohol (low-hi) 


Reading, Recreation, 
Education (low-hi) 


Table 3: Average In-Home Energy Expenditures, 1974-78 


Average $ in-home eneray expenditure 451 764 +69 
AD 12a +66 
sOne .29 +53 


Average $/room in-home energy expenditure 


Average $/room/dd in-home energy expenditure 


= 5 = 
Table 4: Regression Analysis Results - Model 1 -~ $In—Home Enerqy 


eee 


1974 1974 1978 
Unweighted Weighted Unweighted 
ee ee Ss 
Intercept 1ST oo A 225.4 A 298.0 A 
No. of Rooms 3.9 A $20 4A 4.2.2 
CIV o= ot. 2.J0bn's 193 2A 204.9 A 341.1 A 
Halifax Toa is 62. B 162.0 A 
Montreal 1? ez, (vas -16.6 
Toronto -10.2 -3.9 50.5 
Winnipeg -127.1 A -125.4 A =i2 2s 
Edmonton -244.9 A -243.2 A -195.8 A 
Vancouver -22.9 -17.5 -71.9 C 
Total Expenditures .006 A -ON6 A 01 
Age of Head 1 sa N.8 B 3.6 A 
Family Size 13.2 A 12.1 21.6 B 
Education of Head ary N.6 -3.6 
House Type - Semi Det. -50.9 B -49.0 A -23.8 
Rowhouse -81.2 A -88.9 A -119.7 B 
Duplex -12.3 -13.7 -84.6 C 
Adjusted R? 0.43 0.34 0.38 
F value (prob.) 118.5(. 0001) 79.7(.0001) 74.6(.0001 ) 
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Note: A = prob. less than .09M1, B = prob. less than .001, C = prob. less 
than .91 
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Table 5: Regression Analysis Results - Model 2 - $%/Room 
1974 1974 1978 
Unweighted Weighted Unweighted 
Intercept 76.2 A 77.3 A 99.8 A 
City - St. John's 3N.4 A 52.0 A 74.8 A 
Halifax 16.8 A (Meveeb> a] Us A 
Montreal Lap) Gad 6.0 
Toronto -3.5 -1.7 TOs 
Winnipeg -17.6 A -16.3 A -N.9 
Edmonton -37.9 A -36.8 A -26.4 A 
Vancouver Ore 0.8 -6./7 
Total Expenditures P2107 |B 2.5x107 ' B 6toxIn-* A 
Age of Head Ode -. 03 C255 5 
Family Size 0.6 0.04 -0.63 
Education of Head =I A -1.4 B -4.0 A 
House Type - Semi Det. -6.5 C -7.1 8 Bn 
Rowhouse -11.5 A -11.8 A -11.0 
Duplex A PAP 6.676 Dice 
Adjusted R* 31 a9 24 
F value (Prob.) 73.85(.0001) 38.9(.00N1) 41.4(.0001) 
Note: A = prob. less than .0001, B = prob. less than .001, = prob. less 
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Table 6: Regression Analysis Results - Model 3 — $/Room/DD 
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1974 1974 1978 
Unweighted Weighted Unweighted 
ene ee eer 2 hE regiben st) Pe ent. cove loset) girl! s 
Intercept i fae -019 A -2 A 
Total Expenditures 8.01x10-8 B 9.4x10-8 A 1.4x1077 A 
Age of Head 1.8x10-° ~7.6%40-° 9.9x10-> A 
Family Size -1.4x10-5 ~18.4x10-> 27.0x1n-> 
Education of Head 5 SKI A ~4.7x10-* A -7.8x10-* 8 
House Type - Semi Det. 3.4x10-" ees 1D 24,.8x1n-* 
Rowhouse -23x10-"C -35.9x10-+ A -38.8x107* B 
Duplex 16.9x1n-" 6.3x10-" 11.6x107" 
Adjusted R* 01 02 03 
F value (Prob.) 5.6(.9001 ) 6.6(.0001) 9.5(.0001) 


Note: A = prob. less than .0001, B = prob. less than .0M1, C = prob. less 
than .01 
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LOGISTIC REGRESSION ANALYSIS OF LABOUR FORCE SURVEY DATA 


S. Kumar and J.N.K. Rao! 


Standard chisquared (X*) or likelihood ratio (G*) tests for logistic 
regression analysis, involving a binary response variable, are adjust- 
ed to take account of the survey design. The adjustments are based on 
certain generalized desiqn effects. The adjusted statistics are uti- 
lized to analyse some data from the October 1980 Canadian Labour Force 
Survey (LFS). The Wald statistic , which also takes the survey design 
into account, is also examined for goodness-of-fit of the model and 
for testing hypotheses on the parameters of the assumed model. Loqis- 
tic regression diagnostics to detect any outlying cell proportions in 
the table and influential points in the factor space are applied to 
the LFS data, after making necessary adjustments to account for the 
survey design. 


1. INTRODUCTION 


Logistic regression models have been extensively used by researchers in 
social, behavioural and health sciences to analyse the variation in binomial 
proportions (see, for example, the books by Cox (1970) and McCullagh and 
Nelder (1983)). Due to clustering and stratification used in the survey 
desiqn the statistical methods for binomial proportions, however, are often 
inappropriate for analysing sample survey data. For instance, the standard 
chisquared (X*) or the likelihood ratio (G*) tests areatly inflate the type I 
error rate (significance level). Hence, some adjustments to the classical 
methods that take account of the survey desiqn are necessary in order to make 
valid inferences from survey data. In this article, we have utilized two 
simple adjustments to X* or Go based on certain generalized design effects 
(deffs) to analyse some data from the October 1980 Canadian Labour Force 
Survey (LFS) (Section 3). The Wald statistic, which also takes the survey 


design into account, is also examined. 
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In addition to formal statistical tests, it is essential to develop 
diagnostic procedures to detect any outlying cell proportions and influential 
points in the factor space. Regression diagnostics for the standard linear 
model have been extensively investigated in the literature (see the recent 
book by Cook and Weisberg (1982)). Pregibon (1981) recently developed similar 
methods for the logistic reqression with binomial proportions. In Section 4 
some of these methods have been applied to the October 1980 LFS data, after 


making necessary adjustments to account for the survey design. 


2. THEORETICAL RESULTS 


Suppose that the population of interest is partitioned into I cells (do- 
mains) according to the levels of one or more factors, and N, denotes the sur- 
vey estimate of the i-th domain size, Ns Ci 1y. 2, tam ELS N, = N). The 
corresponding estimate of the i-th domain total, Nias of a binary (0, 1) re- 


sponse variable is denoted by N. The ratio estimate, p, = Ni4/Ni5 is used 


1° 
to estimate the population proportion ™ = N.4/N;- 


A logit model on the proportions T; is given by ™ = F (B), where 


In{f,/(1 - f,)} = logit PERE Rel ee ates (1) 


ine< 1). Xi is an s-vector of known constants derived from the factor levels 
and 8 is the s-vector of unknown parameters. Under independent binomial 
sampling in each domain, the maximum likelihood estimates (m.l.e.) are 


obtained from the following likelihood equations: 


X'D(n/n)f = X'D(n/n)as, (2) 


where X! = (x49 nadia Xp» D(n/n) = diag(n,/n, ALY n1/n)s £ 2)f(8)G= (Fay ents 
F,)', and q is the vector of sample proportion q; = n.,/n;, where n, is the 


- ‘ : i ¢ we do 
sample size from i-th domain (Zn, = n). For aeneral sample designs, 


not have m.l.e. due to difficulties in obtainina appropriate likelihood func- 


aur " Moff or. £ 
tions. Hence, it is a common practice to use a pseudo m.l.e." of 8 r 
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obtained from (2) by replacing n; /n by the estimated domain relative size, 


w= N. /M, and q; by the survey See incts P, 


X'D(w)F = X'D(w)p. (3) 


The resulting estimates, B and f =— UB), are asymptotically (i.e., in large 


samples) consistent. The equations (3) may also be written as 
X'Ny(m) = X'Ny, (4) 


where Ny is the vector of estimated counts Nea and N,, (mm) is the vector of 


pseudo elses: Nim) = Nifj> of the totals Nae 


and N,(m), are obtained from (3) or (4) by iterative calculations. 


The estimates B, and hence f 


2.1 Estimated Variances and Covariances 


Let V denote phe estimated covariance matrix of Ps then the estimated cova- 


riance matrix of B is given by 
D(B) = (X'AX)~?(X"D(w) VD(wW)X) (Xt AX)~? (5) 


in larqe samples, where A = diag(w1F1(1 - foe Aes why (1 - fy )). The diago- 


nal elements of (5) provide the estimated variances of the estimates Bs. Sim- 
ilarly, the estimated covariance matrix of the residual vector Ts i) ahs is 


given by 

D(r) = AVA’, (6) 
where 

A = I ~ D(£)D(1 - £)X(X"AX)~'X'D(w). (7) 


The diagonal elements Vs) of (6) lead to standardized residuals r./s.e.(r,) 


which are useful in detecting outlying cell proportions. 


=. bo? = 


2.2 Goodness-of-Fit Tests 


The standard chi-squared test of goodness-of-fit of the model (1) 


is qiven 
by 
tae rey we fy 
Moen Zz, A ee X*. (8) 
ceeee ier.) 2! 
i 
The likelihood ratio test statistic is aiven by 
2. I Z Ps m (1 = Py) I ; b 
eae Wwe pine (1.=p.) In Go ere (9) 
i=1 34 u ¢ 1 L215 .* 
fi. (1 - f.) 
al i 
Note that GS is also defined at p; = 9 and 1 as given by -2nw; In(4 - By 


and -2nw , In fe respectively. Under independent binomial sampling. it is well] 
known that both ei and G2 are asymptotically distributed as a y2 variable with 
I - s degrees of freedom, but for general desians this result is no lonqei 
valid. In fact. X* (or G*) is asymptotically distributed as a weiahted sum 
va 852, . of independent x2 variables, Z., each with 1 d.f. where the weights 
Fi (i = 1. .... I - s) are the eigenvalues of a "qeneralized desiqn effects" 
matrix given by ee Zo - where 


Ey = G'D(f)-'D(1 - £)-*vO(£)-"D(1 - £)-'S, (10) 
tp = Gta-le (11) 
n 


and G is any Ix(I - s) matrix of rank I - s such that G'X = N. i.e.. G is 


: al Feed 
orthogonal to X. Under binomial samplina, =~, } reduces to I. the identity 


matrix 
A simple adjustment to x2 (or G2) is obtained (Roberts. 1984) by treating 


Poetx 76. or CG = 64/8. as 2 with I - s dearees of freedom (d.f.) under the 
c eC 


hypothesis that the model is true. where 


BOGE te 


(eereier =n Vv. (rdw, /TF, J epi (12) 
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The adjusted statistic x? (or G*) should be satisfactory excepting in those 
cases with a large coefficient of variation (C.V.) of the 6; 's. A better ad- 
justment, based on the Satterthwaite approximation, treats xg = 2/1 + a*) or 


Gé = Ge/(1 + a“) as Nae with (I = s)/(1 + An) d.f., where 


at = £ (6, - 6,)°/[(I - 8) 821 (13) 


is the (C.V.)?2 of the 6.'s and 


Teal ae x ‘ 
Bhs 2 OE Vis(r)(nwy)(nm5) / CAFS - FAC - FT, (14) 


a ee Ea 


where Vy?) is the (i, j)-th element of D(r). The statistics Xe and Ge take 
account of the variation in é.'s. 


A Wald statistic for goodness-of fit of the model (1) is given by 


ute che (15) 


where y is the vector of loqits v; =e) Log Lt ae The statistic Xi is distribut- 
ed as 2 with I - s d.f., in large samples. The statistic i is not defined 
af P; = or 1 for some i. Moreover, it becomes unstable when any Pp; is close 
to 1 (see Section 3), or when the deqrees of freedom for V is not larae com- 


pared to I - s (Fay, 1983). 


2.35 Nested Hypothesis 


Suppose the matrix X is partitioned as (X1, X2) where X, is Ixr and X92 is 


Ixu (r + u = s), then the model (1) may be written as 


& = XB = X1B1 + X2Bo, (16) 


= ae 


where 8) is rx1 and 8) is ux1. We are often interested in testing the nul! 
hypothesis H: 8» = 0 given the model (16). The "pseudo m.l.e." under H can be 


obtained from the equations 
, a F A 
X,D(w) f = X,D(w)p (17) 


again by iterative calculations, where £ = f(8). The standard chisquared and 


likelihood ratio tests of H: B 5 = 0 are given by 


Ta watfene fo )* 
ARP. i a ee» ee (18) 
sf gh 
it aes} 
and 
I ¥e fs a os ee f.) | 
G*(2{1) Sac MW tL eee Cea ee ti Oe (19) 
Te | ol ear q a 
f. sae) 
i i 


rel 
of a 


respectively. Under binomial sampling, both x? (21) and G (2{1) are asympto- 
tically distributed as x? with u d.f. when H is true, but for general desiaqns 
this result is no longer valid. In fact x (2/1) or G*(2|1) is asymptotically 
distributed as a weighted sum, r6.(H)Z;, of independent . variables Z., where 


the weights 6 (H) (i =1, ..-, u) are the eigenvalues of the desiaqn effects 


matrix. 


(X3 AR )~ 1 (XSD(w) VD(w) Xp), (20) 
where 


Xo = [LT = X1(X1aX 1) Xt AlXa, (21) 


(Roberts, 1984). In the binomial case, the desiqn effects matrix (20) reduces 
to I, as in the previous case of qoodness-of-fit. 

A simple adjustment to Olt or G7(2|1) is obtained by treatina 
= x*(2]1)/6, (H) or G*(2|1)/6,(H) as y° with u d.f. under H, where 


x2 (7/1) 


GB = 
z V au F.) (22) 
His Ceca Siig) 2» V. (rdw / ee 


and a) is the i-th diagonal element of the covariance matrix of residuals, 


r, (H) =f es f.. given by 


V(r) = DCED(L - AKAK PCE) - £) (23) 
where 
A = (XL0%2)7 PXED Cw) WCW) Ko (KE Aka) * (24) 
A zn a, iy 
The standardized residuals (f. = AT heey 1 can also be computed. As in 


the case of goodness-of-fit. improved approximation. based on Satterthwaite's 
method, can also be obtained. 


A Wald statistic of H 8» = 0 is given by 


X2(2]1) = Bi[D( 82) ]- "Bo. ) (25) 
where D(B2) is the principal submatrix in (5) corresponding to Bo. Under H. 


x*(2]1) is asymptotically distributed as x2 with u df. Inv pactacular Meite p> 
is a scalar. we can treat 89/s.e.( 82) as N(N.1)-variate under the hypothesis 


smeeoe> ULor B°/var( Bp) as Nia Walton diacl mata 


2.4 Diaqnostics 


It is desirable to make a critical assessment of the logit fit by identify- 
ing any outlying cell proportions and influential points in the factor space. 
For this purpose. the vector of residuals and a projection matrix in the 
factor space provide useful tools. However. unlike in the case of the stan- 
dard linear model. the residuals can be defined on different scales The 
natural choice that takes account of the survey desian is the vector of stan- 


a al 
dardized residuals e, = E/N at) |, given in section 2.1. Since the e,'s are 


- 69 - 


approximately N(M, 1) under the model (1), the expected numbers of residuals 
e; exceeding 1.96, 2.33 and 2.58 in magnitude are 0.051, 0.021 and 0.011 res- 
pectively, where I is the number of residuals (cells). These expected numbers 
provide a rough quide to identify any outlying cells. Iqnoring the desiqn and 
hence using standardized residuals under binomial sampling could lead to mis- 
leading conclusions. 

The standardized residuals Cis however, become unreliable for those cells 
with pi = twor close «to 1. pgs Pregibon (1981), we suggest the use of 
components of x? or Ge viz., X, = X / 8. or G. - G./8*, ti TAwidlg Let BE Tee 
Sidual analysis in order to circumvent this difficulty. In either case, larae 
individual components should roughly indicate cells poorly accounted for by 
the model. Index plots (i.e., plots of x. vs i and G. vs i) are useful for 
displaying these components. Normal probabilities plot of X; or G. (i.e., the 
ordered values plotted against standard normal quantiles) is also useful to 
detect deviations from the model (i.e., deviations from a straight-line 
confiquration). 

Pregibon (1981) suqaested the use of diagonal elements, Maas of the pro- 


jection matrix 


i 
5 


al a A 
= 2 eagll 
Mos b= Ve XOX" VEX) X'VE 


I - H (say) (26) 


to detect influential points, where Vh is the estimated covariance matrix un- 


der binomial sampling, viz., diag(p 1(1 - p1)/(nw1), it py ~ py)/(nw,)] in 
the context of survey data. The matrix M arises naturally in solving likeli- 
hood equations (4) by iteratively reweighted least squares, and small values 
of m.. call attention to extreme points in the factor space. Again, an index 


plot (ms; vs i) would provide a useful display. It may be noted that the de- 


; : : si 
sign effect does not come into picture with m,; since we are using "pseudo 


m.l.e." based on binomial sampling. Another useful plot which effectively 
summarizes the information in the index plots x. vs i and mis 


ue ae 
the scatter plot of ths = x?/x* vs hi,» where h.; is the i th diagonal ele 


vs i is given by 


ment of H given by (26) (see Pregibon, 1981). 


280). 


The diagnostic measures eis xX, or G. and Mig are useful for detectina 
extreme points, but not for assessing their impact on various aspects of the 
fit including parameter estimates, 8, fitted values, f, and gqoodness-of-fit 
measures Xe. or G7" or others. Following Pregibon (1981) we suggest three 


measures which quantify the effect of extreme cells (points) on the fit. 


(1) Coefficient sensitivity: Let B. ate 2) denote the pseudo m.l.e. of Bs op 
tained after deleting the &%-th anny data. Then the quantity A. 3) = 
(8, ~ 8, (-2)]/s.e. (8, ) provides a measure of the j-th coefficient Bei Laity 
ee L- th point. jhe ie plots A. (2) vs & for each j provide useful displays 
but the task of looking at the eae plots could become unmanageable if the 
number of coefficients in the model is large. 

(2) Sensitivity of fitted values: Significant changes in coefficient es- 
timates when oe th point (cell) deleted does not necessarily imply that the 
Fitted values f also vary significantly from ‘ale 2), the vector of fitted va- 
lues obtained after deleting the &-th cell, i.e., lf ets could be small. 
We therefore use ies = GAG 1/6. or [x2 » Re) V/S. to assess the impact of 
the 2-th point on the fitted values. where PED) and SESS are given by (9) 
and (8) respectively when i = (8) is replaced by F, (-2) = fF (B(-2)). 


(3) Goodness-of-fit: A measure of qoodness-of-fit sensitivity is given by 
[G* - 67(-2)1/6. or (xX? - X2(-2)16., where G2(-8) “and X7(2%) are the likeli-= 
hood ratio and chisquared statistics obtained after deleting the &-th cell. 
(Note that G2(-2) # 87(-2)). 


3. APPLICATION TO LFS 


We have applied the previous methods to some data from the October 1980 
Canadian Labour Force Survey (LFS). The sample consisted of males aqed 15-64 
who were in the labour force and not full-time students. We have chosen two 
factors, age and education, to explain the variation in unemployment rates via 
logit models. Aqe-qroup levels were formed by dividing the interval [15, 64] 
into ten groups with the j-th age group being the interval [10 + 5j, 14 + 5j], 
mae, 256... 10), and thenwwsing thesmid=poink ofueach interval, A. A etsy elas, 


value of the age for all persons in that age qroup. Similarly, the levels of 


a . 


education. EL were formed by assigning to each person a value based on the 
median years of schooling resulting in the following six levels = 7, 1M, 12. 
(Sapnl4seand) 462 Thus the age by education cross-classification provided a 
two-way table of I = 60 cell proportions. Tap: 

The LFS design employed stratified miibi-4bade cluster sampling with two 
stages in the self-representing (SR) urban areas and three or four staaes in 
non-self-representing (NSR) areas in each province The survey estimates. 
Pa were adjusted for post-stratification. using the projected census age- 
Sex distribution at the provincial level, The estimated covariance matrix 
V of the estimates D. ik is based on more than 450 first-stage units (psu's) so 


that the degrees of freedom for V are large compared to I = 60. 


3.1 Formal Tests of Hypotheses. 


Scatter plot of the logits v., vs age levels A. at each education level F& 


ik j k 
indicated that ey for given k generally increases with age to a maximum and 
then decreases (i.e., the qraph is convex and upward to a maximum). Hence. 


the following model might be suitable to explain the variation in Ts) '8° 


bi : a 2. aes 2 
es In pee Bo + BiA + BoA. + B3E, + Bye, - 
yg ees (5G ede CORE ae (27) 


Some previous work in sociological literature also supports such a mode] 
(Bloch and Smith, 1977). Applyina the results of Section 2 we obtained the 


following values for goodness-of-fit statistics 


yee 496.9 G2 = 101.2 
of 0 er 52S B2/6 = 53.7. 6. = 1-88; 
o ; 2 
Since X* or G* is laraer than Xp 95699) - 73.3, the upper 5% point of x 
with I - s = 55 d.f.- we would reject the model if the survey desian is 


ignored. On the other hand. the value of x2/6 or G2/8. indicate that the mo- 


del is adequate. the significance level (or P-value) beina approximately equal 


90-2 


to 0.52. The value of e when adjusted to refer to xmas) is equal to 47.7 
which is also not significant. Moreover. in the present context with s(= 5) 
relatively small compared to I(= 60), the simple correction d, the average 
cell deff, (see Fellegi, 1980), is very close to See Gime Oa 9905 compared to 
one 1.88: see Rao and Scott (1984) for a theoretical explanation. 

: The Wald statistic xi is not defined here since two of the cells have 
Pik 2 1, but we made minor perturbations to the estimated counts to ensure 
that Pik < 1 for all cells and then computed Ke The resulting values of Xa 
are all large compared to X7/68. (at least 30 times larger than x7/8.) and vary 
considerably (1715 to 3061). Hence, the Wald statistic is very unstable for 


goodness-of-fit test in the present context. If the two cells having Pir =ou 


are deleted, then Xj = 68.4 < xq po(53) = 71.0, indicating that the model (27) 
is adequate. However, it is not a good practice to delete cells just to 
2 


accomodate a chosen test statistic. The other problem with XW noted by Fay 
(1983), does not arise here since d.f. for V is large compared to the number 
of cells in the table. 

The pseudo m.l.e., their s.e. and the corresponding s.e. under binomial 
sampling, all obtained under the model (27), are given in Table 1 along with 
Wald statistic xe (2|1) and G* statistic G*(2/1)/8,(H) for the hypotheses 
H,: B; = 0, i=1, 2, 3, 4 given the model (27). As expected, the true s.e.'s 
are larger than the corresponding binomial s.e.'s. The hypothesis Hy: By, = 0 
(i.e., coefficient of Ee is zero) is not rejected at the 5% level either by 
the Wald statistic or gs statistic. On the other hand, the coefficient, 
Bo, of AS is highly significant. In testing the significance of individual 
coefficients we compare the values of x, (2|1) or G*(2|1)/6_ (H) to Koren) = 
3.84, the upper 5% point of x? - variate with 1 d.f. 

We have also tested the following nested hypotheses given model (27): H3,4° 
B3 = By = 9 (i.e., no education effect); Hoy: Bo = By = O (i.e., no quadratic 
effects). Both H3, and Hoy are highly significant: 


G*(2/1)/8,(H3y) = 282.2/1.64 = 172.1, xy (2]1) = 165.6 for H3,4: 


G*(2|1)/8, (Hoy) = 242.2/2.28 = 106.3, Xy(2|1) = 162.1 for Hz, compared to 
2 
X9..95 (2) = S686 


=. HS 


Table 1: Pseudo m.l.e. B., S.e. (B-), xy(2|1) = B?/var (B.) and G*(2|1)/8 (H.) 
oy 
Values for the LFS Data under Model ‘29% 


i a a OS a eck 


s.e.(B.) 
_ pom tee bd y 1 guy xn (2]1) G°(2]1)/6,(H.) 
i True Binomial . 
0 -2.76 ep 1 24.6 
1 0.209 5031 32 N.n12 250.6 168.4 
z -0.00217 0.0001 73 0.9001 36 TAT 33 107.1 
3 0.50913 0.0891 N.068 5 eg! pee i 
4 0.00276 0.90411 0.0030 N.45 N. 46 


Unlike in the case of goodness-of-fit, the Wald statistics is stable for 
testing nested hypotheses and leads to values close to the corresponding 
G*(2|1)/6,(H) values. 

By the above test of qoodness-of-fit and tests of nested hypotheses we have 


arrived at the following simple model involving only four parameters: 


To. 
Me tee alin vou Jis), = Bo + BJA. + BoA + B3E,, (28) 
Jk ace i i k 
A 


with Bg = -3.10, 8; = 0.211, Bo = -0.09218 and 83 = 1.159 and corresponding 
standard errors are 0.247, 0.0130, 0.000172, and 0.0115. We will use the 


model (28) in Section 3.2 to develop logistic reqression diaanostics. 


3.2 Diagnostics 


We now illustrate the use of diagnostics developed in Section 2.4. 


(i) Residual Analysis 
The 60 cells in the two-way table were numbered lexicographically, and the 


standardized residuals e. were computed under the model (28) arrived through 
i 


ae ee 


formal testing of hypotheses. Amonq the sixty eis cells numbered 6 and 54 
with Par = 1 lead to very large e. values: 166.6 and 6.2 respectively. Amonq 
the remaining eis the residuals numbers 7, 27 and 59 have values 3.84, 
2.73 and 2.52 respectively, whereas the expected number of le; | exceeding 2.33 
under model (28) is roughly 0.02 x 60 = 1.2. Hence, there is some indication 
that cells 7 and 27 could correspond to outlying cell proportions. 

The normal probability plot of G, is displayed in FIG. 1; the plot of X} is 
not given to save space since it is similar to the plot of G;- Figure 1 indi- 
cates no strong deviations from a straight line configuration. The index plot 
of G Figure 2, is consistent with Figure 1. Hence, there is no evidence of 
outlying cell proportions when the components G; of G2 are used for residual 
analysis. 

(ii) Detection of Influential Cells. 

The index plot of mia is displayed in Fiqure 3 which clearly points to 
cells 1 and 6. Fiqure 4 displays the plot of XG /x2 = ene vs Nig wnere the 
line with slope - 1 is given by x? /x? + hj = Zave(h*, ). Here ht, = h, 
x7/x?, and the values of h* near unity corresponds to cells which are out- 


+ 


lying or influential or both (Preqibon, 1981) and appear above the line in 
Figure 3. It is clear that cells 1 and 6, and to a lesser extent cells 7 and 
58, warrant further examination. 

(iii) Coefficient Sensitivity. 

The index plots for measuring coefficient sensitivity (A, (2) vs £) are dis- 
played in Figures 5, 6, 7, and 8 for Bo, 81, B2 and £3 respectively. (he) 
clear from the plots that cells 2 and 3 cause instability in Bo, ae and Py 
whereas B3 is affected by cell 7. 

(iv) Sensitivity of Fitted Values 

Figure 9 displays the plot of hie = Ese )I/ee = c vs & for assessing the 
impact of individual cells on fitted values. Significant peaks in this figure 
correspond to cells 2 and 3 and to a lesser extent to cell 7. Following Cook 
(1977) and Pregibon (1981), it may be noted that the comparison of c to the 
percentage point of x7(s) (s = 4 in model (28)) gives a rough guide as to 
which contour of the confidence reqion the pseudo m.l.e. is displaced due to 
deletion of the £-th cell. The value c = 2.1 for cell 2 roughly corresponds 


to 78% contour of the confidence region. 


> = 


(v) Goodness-of-fit Sensitivity 

Figure 10 displays the plot of [G* - G*(-2) 1/8. vs &: the plot of 
[x2 = Kee y/s. is similar and hence not displayed but the former plot is 
preferred (Pregibon, 1981). Significant peaks in this figure corresponds to 
cells 2, 3, 7, 27, 39 and 54 (values 2 3), the most significant being cell 7 
with the value 5.4. By deleting cell 7 and recomputing the adjusted statistic 
G°(-2) = G--2)/8 (=2) where 6 (-2%) is the corresponding value of 6 
a value of 48.43 with 55 d.f. compared to G7/8. = 55.3 with 56 d.f. 


.» we qet 

Qur investigation on the whole indicated that cells 7, 2? and 3 are possible 
candidates for deletion, but we feel that their impact is not siqnificant 
enough to warrant their deletion - one would like to explain the variation 


among all cell proportions unless certain cells contribute heavily to the 


disagreement between the data and the fitted model. 
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Figure 7: Index Plot of {Bo - Bo(-2) }/s.e. (Ba) 
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Figure 8: Index Plot of {83 - B3(-2) }/s.e.(B3) 
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APPLICATION OF LINEAR AND LOG-LINEAR MODELS 
TO DATA FROM COMPLEX SAMPLES 


Robert E. Fay! 


Most sample surveys conducted by organizations such as Statistics 
Canada or the U.S. Bureau of the Census employ complex designs. The 
design-based approach to statistical inference, typically the insti- 
tutional standard of inference for simple population statistics such 
as means and totals, may be extended to parameters of analytic mo- 
dels as well. Most of this paper focuses on application of design- 
based inferences to such models, but rationales are offered for use 
of model-based alternatives in some instances, by way of explanation 
for the author's observation that both modes of inference are used 
in practice at his own institution. 


Within the design-based approach to inference, the paper briefly 
describes experience with linear regression analysis. Recently, 
variance computations for a number of surveys of the Census Bureau 
have been implemented throuqh "replicate weighting"; the principal 
application has been for variances of simple statistics, but this 
technique also facilitates variance computation for virtually any 
complex analytic model. Finally, approaches and experience with 
log-linear models are reported. 


1. INTRODUCTION 


Statistics Canada has played a siqnificant role in many of the methodo- 
logical developments in the application of analytic methods to sample survey 
data. The intent of this paper is to review and to share some of the 
experience acquired by the U.S. Bureau of the Census with these same 
questions. 

The "design-based" (also sometimes called "classical") mode of inference 
predominates in the analysis and presentation of data by most qovernmental 
statistical agencies, such as Statistics Canada and the U.S. Bureau of the 


Census, as well as by most large private survey organizations. The basis of 
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statistical inference with this approach is the randomization employed to 
select the sample from the finite population. Construction of confidence 
intervals and tests of hypotheses are based on a laraqe-sample theory tied to 
this randomization rather than to a specific model. Standard texts such as 
those by Cochran [4], Kish [17], and Hansen, Hurwitz, and Madow [14] present 
the elements of this theory. Hansen, Madow and Tepping [15] recently arqued 
the advantages of this approach to the problem of inference from survey data 
over "model-based" methods; Sarndal [25] and Cassel, Sdrndal, and Wretman [3], 
have discussed the choice between the model and design-based approaches from a 
somewhat different point of view. Most of the original development of the 
design-based theory of inference was specifically for population totals, pro- 
portions, means, and ratios, and much of the corresponding literature for the 
model-based theory similarly concentrates on such basic statistics. 

Common analytic models, such as linear regression, log-linear models, and 
generalized linear models, on the other hand, were initially developed in the 
context of explicit stochastic models, for example, the normal or multinomial 
distributions. "Classical" inference here has generally come to refer to sta- 
tistical inferences based upon such distributional assumptions (where "classi- 
cal" may include "Bayesian" in this discussion). Developments in "robust" 
estimation avoid specific distributional requirements, but often maintain 
assumptions not typically encountered in survey sampling, for example, that 
the error terms of the model are independent and selected from a symmetric 
population. 

Many researchers familiar with one or more of these analytic models have 
applied them directly to sample survey data without recognition of the possi- 
ble consequences of the sample design on the validity of inferences based on 
the usual distributional assumptions. The subject of this conference, of 
course, essentially concerns "design-based" alternatives that do reflect the 
effect of the design. Althouah all other sections of this paper will address 
"design-based" methods, the next section considers some of the theoretical and 
practical issues in choosing between these two approaches, and how these con- 
siderations appear manifested in practice at the Census Bureau. 


The third section briefly describes some of our experience at the Census 


Bureau with design-based methods for linear regression. The fourth section 


discusses an approach taken in the computer implementation of replication 
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methods, using "replicate weights". Although principally intended for the 
computation of variance for the usual survey characteristics, this technique 
also facilitates computation of standard errors for complex models. This 
general approach may be particularly useful for less standard models, i.e., 
models other than the linear, log-linear, and other qeneralized linear 
models. Finally, some developments with respect to log-linear models are dis- 


cussed, including specific computer software. 


2. CHOOSING BETWEEN DESIGN-BASED AND MODEL-BASED 
INFERENCE FOR ANALYTIC MODELS 


The choice between design-based and model-based inference may involve 
several factors, including effects of stratification, and existence or extent 
of dependence between sampled values ("clustering"). Many of the essential 
issues related to this general choice are enumerated by DuMouchel and Duncan 


[6] in their discussion of whether to incorporate survey weights in linear 


regression. 
If Y represents a column vector of observations Yes and X = iX, a], Mesa: 15 
-e-, 9 represents predictors for Y, the model . 
Y=XB+E (WY) 


with © = le, } composed of independent, identically distributed error terms 


e.~N(O, ca has as its maximum-likelihood estimate for B 


; ee eal 
ee CE aN, (2!) 


Typical survey estimation associates a weight Ws with each survey case i, 
based on the inverse of the probability of selection, often adjusted by 
factors for nonresponse and ratio estimation. If W represents a diagonal 


matrix of Wis then 


X WX) 1X Wy (e3) 
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gives a desiqn-consistent alternative incorporating the weights. Under the 
original stochastic model justifying the choice of (2.2), or, more aenerally, 
if the E.'s are uncorrelated with zero expectations and equal variances, (2.3) 
has a larger sampling variance than (2.2). On the other hand, if these 
specific assumptions fail (particularly concerning the expectations of the 
e,'s), (2.3) remains a desiqn-consistent estimate of the census parameter, 8*, 
defined as the application of (2.2) to the values in the complete finite popu- 
lation, whereas computation of (2.2) for unweighted sample cases cannot 
guarantee consistent estimation of 8*. 

DuMouchel and Duncan further elaborate on the issue of choosing between the 
variance advantage of (2.2) under the simple model and the consistency of 
(2.3) under model failure. Their presentation includes a number of citations 
to earlier commentary by others on both sides of this controversy, and can be 
recommended for its balanced perspective. Additionally, they propose a test, 
which can be performed with typical computer packages for linear regression, 
of whether the weighted and unweighted reqressions are sianificantly differ- 
ent. If the test rejects the hypothesis that (2.2) and (2.3) are consistent 
estimates of the same set of coefficients, then the argument for consistency 
with the census value, 8*, favors (2.3). If the test does not reject, the 
authors prefer (2.2) with its (generally) lower variance. 

If a researcher rejects (2.2) on the basis of the test proposed by 
DuMouchel and Duncan, and computes (2.3) instead, the implications of this 
choice are relatively clear: that (2.3) is selected over (2.2) for its consis- 
tency under failure of the model. If the test "accepts" the hypothesis, and 
(2.2) is used with its associated standard errors derived under the model, 
caution is nonetheless required in uncritically interpreting (2.2) and associ- 
ated confidence intervals as statements about the census parameter 8*. In 
many applications, choice of (2.3) and its associated reliability could be 
defended as the only "safe" interpretation of the data as an estimate of B* 
when model failure is suspected, in spite of possible acceptance by the test 
of a hypothesis of no significant difference between the weighted and un- 
weighted analyses. 


The paper of DuMouchel and Duncan clearly illustrates the most essential 


consideration in choosing between model-based and design-based inference, 


namely, efficiency under a correctly specified model versus consistency under 
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failure of the assumptions of the model. Two footnotes may be added. Al- 
though ignoring survey weights is inconsistent under any design-based approach 
and can only be justified under model-based approaches, not all model-based 
inference requires ignoring the information represented in the weights. 

Rubin [24] gave a concise explanation of this last point in his discussion 
of the paper of Hansen, Madow, and Tepping [15]. Referring to the more exten- 
sive work of Rosenbaum and Rubin [22], Rubin pointed out that a complete 
Bayesian interpretation of the observed data reflects not only consideration 
of the functional and distributional relationships in the total population 
(such as models like (2.1) for the complete population) but also the process 
by which the sample observations become observed. (In a randomized desigqn, 
"propensity" to be included in the sample may be equated to probability of 
selection and the "propensity score" in Rosenbaum and Rubin [221.) On the 
basis of this consideration, Rubin [23] presented an interesting justifica- 
tion, from a Bayesian perspective, of the use of randomization in sample 
selection, a procedure that has been staunchly defended by proponents of 
design-based inference but treated with some disdain by many proponents of 
model-based inference. Consequently, Rubin advocates model-based inference 
tempered by careful analysis of the effects of selection ,or propensity to be 
included in the sample; these principles in some circumstances could lead to 
either (2.2) or (2.3), or perhaps alternatives to both. 

As a second footnote, DuMouchel and Duncan explicitly restricted their 
attention to the issue of weighting for stratified simple random sampling. An 
equally important issue in many applications is the effect on inferences of 
clustering, that is, dependencies amonq sampled units due to their joint 
inclusion in the sample by design, such as persons in sampled households or 
persons in neighboring households jointly selected into sample. In self- 
weighting samples (where all sample cases have equal weight), design-based and 
model-based analyses may often produce the same estimates of the parameters of 
an analytic model but substantially different assessments of their reliabi- 
lity, unless the dependencies from clustering are explicitly incorporated into 
the model-based inference. Unlike the issue of the use of weights in strati- 
fied simple random samples, where a model-based approach may be defended if 
the error terms conform to the original full specification of the model, a 


known dependence among the observations due to clustering (to any serious 
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degree) inherently conflicts with any assumption of independence of errors 
that might be required by an overly simplified model. Hence, models that do 
not reflect known effects of clustering automatically fail to model the data 
properly. 

Design-based inference is the institutional standard at the U.S. Bureau of 
the Census; yet, practice incorporates both modes of inference with respect to 
models. Researchers are most likely to adhere strictly to a desiqn-based 
standard for inferences to national relationships based upon complex samples. 
When survey weights vary by only a modest deaqree or not at all, and the 
effects of clustering may be presumed small, model-based inferences for ana- 
lytic models appear to enjoy acceptance. The attraction of model-based 
inference in these cases, no doubt, reflects less a philosophic choice than a 
practical one: model-based methods are more accessible and familiar than the 
design-based counterparts. (The author has encountered applications meeting 
such conditions on variation on the weights and effects of clustering where 
design-based methods simply duplicate model-based conclusions, thus justifying 
the substitution of model-based methods under similar favorable circum- 
stances. When the weights do appreciably vary, or characteristics are subject 
to considerable clustering, however, examples are easily found where the two 
modes of inference substantially disagree, and where the model-based inference 
is highly questionnable. ) 

Specific areas of application at the Census Bureau appear almost exclu- 
sively model-based. Methods for imputation of missing data, in particular, 
some of which derive from explicit parametric models, characteristically avoid 
any consideration of design-based weights. Another specific field of study, 
estimation for small areas or domains, often reflects a mixed strategy of 
design- and model-based inference. Thus, practice at the Census Bureau 
appears to parallel the choice outlined by DuMouchel and Puncan: ef ficiency 
(and simplicity) under the assumed model versus consistency under model 
failure. Strict inference to national relationships are most likely to elicit 
design-based methods, while less formal analyses or analyses in which the 


model is hoped correct (missing data) often favor a model-based approach. 
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3. DESIGN-BASED INFERENCE FOR LINEAR REGRESSION 
AT THE U.S. CENSUS BUREAU 


In general statistical practice, linear regression is probably the single 
most popular analytic technique. Most data collected by the Census Rureau, 
particularly for the "demographic areas" involving characteristics of persons 
or housing, are categorical: linear regression, in any form, is used rela- 
tively seldom at the Census Bureau by comparison. 

Fuller [13] developed basic results in desiqn-based inference for linear 
regression, using methods based upon Taylor-series expansions (lineariza- 
tion). These results are incorporated in the computer program SUPER CARP 
[16], whose development was partially supported by the U.S. Bureau of the 
Census. We can report successful use of the program ourselves, although it 
has been applied to only a few problems thus far. The report by Moore [26] is 
probably the most accessible illustration of the use of SUPER CARP at our 
institution. 

The next section discusses the implementation of replication methods 
through replicate weights, and we have qiven preliminary thought, but not yet 
attempted to implement, alternative computer software specifically designed 
for this approach. No substantial philosophic difference with SUPER CARP is 
implied by these considerations, although replication methods tend to qive 
slightly larger and thus more conservative standard errors than lineariza- 
tion. The intent in developing this software would be to take advantage of 
replication methods developed for some of our surveys, which can be made to 
reflect the effects of complex estimators more completely than programs imple- 


menting linearization. 


4. COMPUTING DESIGN-BASED VARIANCES THROUGH REPLICATE WEIGHTS 


Replication methods, such as jackknife, half-sample, and bootstrap tech- 
niques, represent the principal general alternative to linearization for 
design-based variance estimation for nonlinear statistics. Kish and Frankel 
[18] presented an early discussion of the use of replication for such purposes 


and much research has been conducted since. 


The popularity of replication for variance estimation has gone through 
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cycles. Linearization is a powerful technique, of course, and relationships 
presented by Binder [1] facilitate its implementation for a wide class of 
analytic models. Census Bureau surveys tend to employ quite complex estima- 
tors, however, and fully representing the effect on the sampling variances of 
these estimators has frequently proven to consume large amounts of profes- 
Sional time, both by statisticians and, especially, experienced computer 
programmers. Recently, variance computations for a number of surveys have 
used replication methods achieved through a "replicate weighting" approach. 
The principal features of this method are to provide a unified approach to 
enable the computation of variances for a large number of survey characteris- 
tics and to simplify the estimation of variance for complex analytic 
statistics. 

The replicate weighting approach is not a new discovery: some of its 
earlier history is reported in [5], which also describes experience acquired 
by the U.S. Bureau of Labor Statistics, Bureau of the Census, and Westat, 
Inc. The algorithm may be said to represent the variance from a (possibly 
complex) design and a (possibly complex) survey estimator in the form of data 
to be associated with the survey data file rather than as a set of (possibly 
complex) variance formulas requiring computer programming. Familiar replica- 
tion methods, such as balanced half-samples and the jackknife, may be repre- 
sented through replicate weights, but the algorithm also facilitates the 
implementation of a much wider class of resampling plans, as in [7]. In [10], 
it is shown that there exists a resampling plan (actually an infinite number 
of resampling plans) corresponding to essentially any familiar variance esti- 
mator for estimates of population totals, such as variance expressions for 
multi-stage designs, Yates-Grundy estimators, etc. By representing complex 
variance relationships as data, variance computation becomes accessible to a 
larger group of data users. 

Estimation in many surveys assigns weights Wjg to each case i, so that for 
any characteristic Xi estimates of total are given by the weighted sum of the 


characteristic times the survey weight 


Xo = ) WioXi- (4.1) 
nl 
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The product of the replicate weighting approach is a set of additional 
weights Ww r= 1, ..-, R, for each survey case i, from which alternative es- 


timates of total 


ee tse (4.2) 
iL 
may be computed. The estimate of variance is given by 


(X. - X)? (4.3) 


for predetermined d. independent of the choice of survey characteristic X. 
(As an example, a simplified balanced half-sample estimate of variance. 
ignoring the effect of any complex survey estimation reflected in the weights 
Wi0, would be given by assigning weights ee equal either to 2Wjo or to 0 ac- 
cording to whether case i was included in half-sample r, and setting d.. me AR 
for each r.) More generally, for a smooth function S that are functions of 
weighted population estimates of total Kg 6), ees XK) 
Kai 


each of the form 


d coo, +6: eo z s(x$)), od, xgk)) 12 (4.4) 


The estimator S in (4.4) may stand for the sometimes extremely complex estima- 
tors often used in survey estimation, incorporating noninterview adjustments 
and ratio or iterative ratio estimation. Furthermore, these forms of complex 
survey estimation, if incorporated in the weights Wis may be included in the 
derivation of Wie as well. Thus, variance computation with this approach 


falls naturally into three distinct steps or phases: 


1. Generate replicate basic weights ae for the simple unbiased (Horwitz- 
Thompson) weighting of the data given by the basic weights Wjo*. 


2. Compute replicate (final) weights, W. 


a7? by applying the same noninter- 


er as 


view and ratio estimators to the replicate basic weights, We *. ag the 
r 
original estimation procedures used to compute Wj9 from the Wio*. 
3. Apply (4.4) to the estimation of variance of simple or complex 


statistics. 


The modularity of the preceding three phases is a key feature of this tech- 
nique: general programs may be used to perform phases 1 and 2, or custom pro- 
grams may be written to cover unusual circumstances as required. For a single 
survey, phases 1 and 2 need be performed only once. Programs for phase 3 need 
take no specific note of the design or estimator and can be run as_ needed 
by any user with access to the replicate weights Win produced in the second 
phase. 

Although most applications of this method at the Census Rureau have been to 
estimate variances for basic survey characteristics such as means, totals, or 
proportions, (4.4) lends itself well to analytic purposes as well. This 
approach fully represents the effects of complex desiqns and estimators, 
whereas in practice implementation of linearization often is restricted to the 
more common and simple situations. Furthermore, although specific computer 
software may be developed to implement linearization for common analytic 
methods, such as linear regression, log-linear models, generalized linear 
models, etc., formula (4.4) enables researchers to compute variances for more 
specialized analytic models for which no linearization methods have been pro- 
grammed, since (4.4) only requires that the researcher apply complete data 


algorithms to the alternative estimates produced by the replicate weights. 


5. DESIGN-BASED INFERENCE FOR LOG-LINEAR MODELS 


Log-linear models, which express the logarithm of the expected frequencies 
for categorical responses as a linear function of unknown parameters, encom- 
pass both factorial models for cross-classified categorical data, and logistic 
models for one or more dependent categorical variables as a function of any 
combination of categorical and continuous predictors. Bishop, Fienberg, and 
Holland [2] provided one of the earliest books in this rapidly expanding 
field. 
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Many log-linear models, particularly those for fully cross-classified 
categorical data, involve a large number of parameters. The three most 


typical problems of inference are: 


1. To compute standard errors and confidence intervals for the individual 
estimated parameters, 

2. To test the significance of the contribution of specific sets of para- 
meters to the fit of a model, 


3. To test the overall qoodness-of-fit of the model. 


In the context of simple random samples, standard results in maximum like- 
lihood theory provides an answer to these questions, although the Pearson 
chi-square test rightfully enjoys greater popularity than the likelihood-ratio 
chi-square test as a solution to the third problem. 

Koch, Freeman, and Freeman [19] extended the Weighted Least Squares (WLS) 
method to complex samples, thereby providing solutions to each of the three 
principal inferential problems. While this method has proven of substantial 
general use, it is limited in some applications by the necessity to produce 
highly precise estimates of the desiqn-based covariance of the sample esti- 
mates before the asymptotic theory approximates the actual performance of the 
WLS procedures. (Further comments on the limitations of WLS are given in [8] 
and” (14) 

Fellegi [12] made an early contribution to the development of alternative 
tests to WLS for specific situations. More recently, Rao and Scott [20], [21] 
have formulated and extended a set of related methods to cover the problem of 
testing for a general class of models including log-linear models. Develop- 
ment of these methods has been closely associated with Statistics Canada. 

A less well-known "jackknife chi-square test" [11] gives an alternative 
approach to the general problem of design-based tests of hypotheses. This 
test is based upon replication, using (4.4) and a similar expression related 
to the approximation of the first-order bias (as in the usual jackknife) to 
draw approximate inferences about the null hypothesis distribution of the 
usual chi-square tests applied directly to the weighted survey estimates. The 
method shares much in common with those developed by Rao and Scott. Although 


a full comparison of the relative merits the jackknifed test and the tests 
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proposed by Rao and Scott has not been conducted, the preliminary suagestion 
is that both work well and neither entirely dominates the other. (Further 
comments are given in [11].) 

The jackknifed tests do appear somewhat easier to implement, however, 
especially to tables involving a large number of cells. A FORTRAN computer 
program, CPLX (described in [8] and documented by [9]), implementing the jack- 
knifed tests for factorial log-linear models for cross-classified data is now 
in the public domain. The program also computes replication-based standard 
errors for parameters of loq-linear models, thus also addressing the first of 
the three problems of inference listed earlier. Although CPLX fits well into 
an environment in which other survey variances are also estimated through 
replication approaches, such as the replication weighting techniques described 
in the previous section, these circumstances are by no means necessary to use 
the proqram, and a number of researchers within and outside the Census Bureau 
have applied the proqram in a variety of settinas. 

In time, the author hopes to he able to incorporate the methodology of Rao 
and Scott into a proaram like CPLX in order to make both methods available. 
For the short term, however, the current version of CPLX should be of help to 


researchers seeking design-based inferences from survey data. 
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LEAST SQUARES AND RELATED ANALYSES FOR COMPLEX SURVEY DESIGNS 
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1. INTRODUCTION AND MODEL 


Assume that a sample of clusters of elemental units is selected from a 
finite population divided into L strata. The total sample of n clusters 


(primary sampling units) is given by 
L 
ast = ny (1) 


where ny, 2 2 is the number of clusters selected in the h-th stratum. A column 
vector of characteristics 


a tCy . (2) 
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is observed for the j-th elemental unit in the i-th cluster of the h-th stra- 


tum. The vector Y is quite general. For example, some elements of the 


hij 
vector can be the powers of products of other entries. Also, one element can 


be, and often will be, identically equal to one. The cluster totals for the 


vector are defined by 


Mi 


Te (3) 
e Ani 


Lie 


where mis the number of elements in the hi-th cluster. 
We shall be interested in the behavior of locally continuous functions of a 


linear function of the vector of cluster means 


: Wayne A. Fuller. Department of Statistics, Iowa State University. 
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where Wh are fixed weights. Often the weights are 
Ww. = NNT? (5) 
ay oe a : 


where Nu is the number of clusters in the h-th stratum and N is the total num- 
ber of clusters in the population. For the weights (5) the linear function in 
(4) is the usual unbiased estimator of the finite population mean per clus- 
ter. Another set of weights that often is of interest is the set of unit 


weights 
Wie a i heaniynec (6) 


Our model permits us to consider functions of the mean per element. The 
usual estimator of the mean per element for a particular Y-variable is the 
ratio of the mean per cluster for the Y-variable to the mean per cluster of 
the number of elements. The mean number of elements per cluster is the clus- 
ter mean of a Y-variable that is identically one. 

Our discussion can be easily expanded to include various forms of subsam- 
pling within clusters. Because such expansions add little to the generality 
of the discussion and add considerable notational complexity, we restrict our 
attention to single stage sampling within strata. 

Our discussion rests heavily on the following central limit theorem for 


samples from a finite population. 


Theorem 1. Let (Ee r= 1, 2, ...} be a sequence of stratified finite popu 
lations. Let the population in the h-th stratum of the r-th population be a 
random sample of size Neh > ere selected from a p dimensional infinite 
population with absolute 2 + 6, where 6 > 1, moments bounded by Ms q-M lot! priret 
the covariance matrix for the rh-th infinite population be Zone Let L. > Lnu4 


be the number of strata in the finite population and let a simple random 


Hae: 


sample of ny (ny 2 2 andn, 2 eee units be selected in the h-th stra- 


tum. Let fy = Nabn rh De a triangular array such that 


where Me is a fixed number. Let Vieni be the total for the i-th cluster se- 


lected in the h-th stratum for the r-th population and let 
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where ere is the finite population parameter and er is the mean of the in- 
finite population used to generate the h-th stratum of the finite population. 


Assume 


L 
LW, el 
ie. cee 


where the M's are fixed numbers and assume that 
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as r + ©, where Weh is a trianqular array of weights. Then 
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The proof of this theorem follows from Theorems 1 and: 2 of Fuller (1975) 
and can be extended to multistage samples. Also see Krewski and Rao (1981) 


and Isaki and Fuller (1982). 


Most of our applications are to continuous functions of 9. 


Corollary 1. Let the assumptions of Theorem 1 hold. Let g(®) be a vector 
valued function of 8, where g(8) is continuous with continuous first deriva- 
Hives) for 0ain the sphere los &.| = 6 for all r, where 63> 1) “ds fixed hebet 


GC 8) be the nonsingular matrix of first derivatives of q(8), where the ij-th 
element of G(8) is 


dq, (8) 


30. 
j 


9 


q, (8) is the i-th element of q(8) and 2 is the j-th element of 8. Then 
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Corollary 1 is stated for the Taylor estimator of the variance of the approxi- 
mate distribution of a8.) - G8). Suitably defined replication estimators 
of the variance can also be used. Replication methods include balanced repli- 
cation methods (see McCarthy (1969)), jackknife methods (See Miller (1974)) 
and bootstrap methods (see Ffron (1979, 1981)). While these methods can he 
adapted to the sampling situation, the adaptation is not always immediate (see 
Rao and Wu (1983)). 

One class of continuous functions of 8 that deserves special attention is 
that obtained by using 8 as the dependent variable in a qeneralized least 


squares fit. 


Corollary 2. Let the assumptions of Theorem 1 hold. Let 9 satisfy 


Q= A(g)- 


where @ is a k-dimensional vector (k < p), h(a) is a continuous function of gq, 
with continuous first and second derivatives for all @ in an open sphere 
containing the true a. for’ all =r. Lew the parameter space for a be an open 
bounded subset of k-dimensional Euclidean space. Let a. be the vector that 


minimizes 


Then 


where 


- 102 - 


and H(a,) is the matrix of first derivatives of h(a) with respect to a evalu- 


ated at a. 


2. MEANS, RATIOS AND REGRESSIONS 


An elementary application of Theorem 1 is the estimation of the mean per 
cluster and the setting of approximate confidence limits for the mean per 
cluster. Often the parameter of interest for the mean estimator is the finite 
population mean per cluster, in which case the finite population correction 
Ci re) would be included in the variance estimator. 

A slightly more complex application is the estimation of the difference 


between the means per cluster for two domains. If we let 


YRigt = observation on characteristic of interest if element hij is in 
domain 1 
= O otherwise, 
"ni j2 = observation on characteristic of interest if element hij is in 
domain 2 
= O otherwise, 
is = 1 if element hij is in domain 1 
= 0 otherwise, 
each = 1 if element hij is in domain 2 


= O otherwise. 


the estimated difference between the mean per element in the two domains is 


A 


OL ne = Daan © Uap (7) 


~~ 


Two methods of computing the Taylor estimator of variance are often used. The 
first method computes the estimator of Corollary 1 directly from the matrices 
G8) and vig, ~ a4 or Vig, ~ al An algebraically identical computational 


procedure is to define the observations 
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and to compute the ordinary stratified estimator of the variance of the mean 


per cluster for Z. 5. 


Be = age, ; g 
where 
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For example, the computational form (9) is used in Super Carp. See Hidiroglou 
etal. (1980, p. 32). 

The analyst may be interested in inferences for the particular finite popu- 
lation sampled or for the superpopulation when working with quantities such as 
differences of means. 

One of the more frequent analytic uses of survey data is the computation of 
regression equations. In fact, the difference between domain means can be 
expressed as a regression coefficient. Although the vector of regression 
coefficients is of the form a(8) described in the previous section. it may be 
advantageous to partition the Y-vector of Section 1 into several parts and to 
give the reqression coefficients explicit expressions. The reqression equa- 


tion can be written as 


Ve NCE Ei Re oa Pe a (10) 


. is the dependent variable, the vector Xn is a k-dimensional 


where “hij j 
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vector of explanatory variables. The weiqhted least squares estimator of B is 
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The weights Nha are permitted to be a function of hij, but we will assume 
that the weights are fixed in the sense that they depend only on the elemental 
identification. This precludes from consideration (except as an approxima- 
tion) the use of weights that are a function of other elements entering the | 
sample. 

Under mild assumptions on the moments of the superpopulation generating the 
finite population, Theorem 1 is applicable to the estimator defined in (11). 
hij? then the estimator By is a 
consistent estimator of the finite population vector 


If the selection probabilities are denoted by 17 


B [ : > “hi Keel yee if “hi Y (12) 
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It follows from (12) that the estimator (11) is a consistent estimator of the 


finite population regression coefficient when W is proportional to the in- 


hij 
verse of the selection probabilities. The error in By as an estimator of Be 
is 
agains aE oa 
= = DG on AN enero CLIN a Dy »y ran ran Ane 
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By Theorem 1 and Corollary 1 a consistent estimator of the variance of the ap- 


proximate distribution of By - 8 is 


Wilgewer eg) SHA GAT. (14) 
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and 8 is the superpopulation analog of Bos This particular form of the esti- 
mator of variance was suqgested by Fuller (1975) and is used in Super Carp. 
One of the frequently asked questions faced by survey statisticians is: "In 
computing the reqression equation, should I use the sampling weights?" As 
with most such questions, the answer is "It depends." The fact that the 
question is asked generally means that the aquestionner has in mind inference 
for a population beyond the finite population sampled. This does not mean 
that the particular superpopulation is completely defined or definable. It 
does suagest that the questionner is postulating that the finite population is 
generated by a superpopulation in which some type of linear model holds. (ne 
quantification of the hypothesis that weights are not required is_ the 


superpopulation hypothesis 


2 oh (15) 


where the 8's are superpopulation analogs of (12), 
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and E, denotes expectation with respect to the superpopulation. This is a 
testable hypothesis. It seems that, at a minimum, a test of this hypothesis 
should be constructed if one performs an unweighted analysis of a sample with 
unequal selection probabilities. 

If the null hypothesis also includes the hypothesis that the estimator with 
unit weights is the minimum variance estimator, then the test of the hypothe- 


sis is given by the statistic 


k op laeeae ie 
Sim a (17) 
where 
Cy el ake eae : ee 
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is defined by (14) with Zi, replacing X As the notation suqgests, the 


ae hay as 
statistic is approximately distributed as Snedecor's F with k and n - L - 2k 


deqrees of freedom. 


Example 1. Table 1 contains observations on 37 area seqments collected by the 
Statistical Reporting Service, J.S. Department of Agriculture in northcentral 
Iowa in 1978. Two determinations on the hectares of soybeans are reported. 
The first is obtained by personal interview in the June Enumerative Survey. 
The second is obtained from a classification of Landsat data based upon a 
classifier developed by the Statistical Reporting Service. The original ob- 


jective of the study was to use the Landsat data to construct a reqression 
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estimator of the total acres. We use the data to illustrate the computation 
of regression statistics from survey data. The sample most nearly approxi- 
mates a stratified sample with strata identified in the column headed 
"county". The inverse of the sampling rates is given in the weight column. 
The estimated reqression equation for the regression of interview hectares on 


satellite hectares defined by estimator (11) is 


Y = -11.845 + 1.1602x, 
(8.332) (0.0922) 


where the numbers in parentheses are the standard errors obtained from the 
estimated covariance matrix calculated by equation (14). 

Calculations were performed using Super Carp. If the equation and standard 
errors are calculated using unit weights in equations (11) and (14), respec- 


tively, we have 


Y See 72i + 1 .0800X. 
9282 ei 00963) 


If we calculate the F-test suggested in equation (17), we obtain 
Zz 
Fo, = 2.81. 


At first glance, this test is large enough to cause to suspicion about the 
equality of the two coefficients. Because this sample is very small and 
because of the structure of the weights, the test is nearly a test between two 
lines, the line for county one, and the average line for the remaining 
counties. In this small sample the deviations from the line in county one are 
small. Hence, the estimated standard errors of the coefficients for the two 
added variables are small. This phenomenon is discussed further in Section 
3. If one uses the ordinary rearession F-test that assumes homogeneous error 


variances and ignores the stratification, one obtains 


2 
Fo, = 0.68. 
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While this statistic is not distributed as Snedecor's F, it does make one 
feel more comfortable with the assumption that the two weighting procedures 
are estimating the same equation. 

Table 2 contains the standard errors of regression coefficients estimated 
under alternative assumptions. The estimated standard errors: for the inter- 
cept behave much as one might anticipate. The stratified weighted sample pro- 
cedure has the smallest estimated standard error followed by the stratified 
unit weight procedure and the ordinary least squares procedure. Do not forget 
these are estimated standard errors. MTJhe two stratified procedures are con- 
sistent under the stratified model. The weighted estimator has smaller 
variance because the observations for stratum 1, the stratum with the largest 
weight, lie closer to the estimated line than do the points in other strata. 
The ordinary least squares estimated standard error is not consistent under 
the stratified model. If the sample is treated as a cluster sample of coun- 
ties, the estimated standard errors for the intercept are about 3N to 4N 
percent larger than the corresponding values for the stratified sample. 

The estimated standard errors for the slope display a different behavior. 
The smallest estimated standard error is associated with the unit weight clus- 
ter estimation, and the largest estimated standard error is associated with 
ordinary least squares. Roughly speaking, the variation of slopes among clus- 
ters is small relative to the within cluster variation. Because the weights 
are inversely correlated with the observed variability, the weighted estima- 
tors have smaller estimated variances. This is a small sample, but it is 
sufficient to demonstrate that unit weights do not always produce smaller 
variances than sample weights and that stratification and clustering can have 
rather complex effects on the estimated variances of the regression 


coefficients. 


3. WHAT IS A LARGE SAMPLE? 


Our discussion has rested on the large sample properties of estimators and 
of estimators of variance. If the limiting normal distribution is being used 
to establish confidence intervals, the size of the sample required for a qood 


approximation depends upon the nature of the original population. For 
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example, if the characteristic is a rare zero-one item (probability less than 
0.05, say), a very large sample (more than 1,400 for a simple random sample 
(Cochran, 1977, p. 58)) will be required for the normal approximation. The 
binomial with small p is only one example of the very skewed populations often 
encountered in sampling practice. Measures of size such as gross sales of 
firms, number of employees of firms, number of animals per farm, and family 
income are examples of skewed populations for which large samples are required 
before the distribution of the mean approaches normality. On the other hand, 
the distribution of the mean for items such as family size may approximate the 
normal distribution for small (less than 100) sample sizes. 

The use of the Taylor expansion is semi-nonparametric in that the approxi- 
mation holds, in large samples, under very mild assumptions on the popula- 
tion. The large sample requirements are met if we have no isolated points in 
our sample space. The method may perform poorly in situations where the qgene- 
rating distribution and sample size are such that an observation or observa- 
tions are isolated from the remaining cluster of points. We consider the 
problem of estimating the variance of the vector of regression coefficients 
used to test the effect of weighting on the coefficients in the soybean 


example. The oriqinal vector is 
(DitoX oe XW 5 JW): 


and the hypothesis to be tested is the hypothesis that the coefficients for XW 
and W are zero. To illustrate the problems associated with variance estima- 
tion for the vector of coefficients for the soybean data set, we create a vec- 
tor that is orthogonal in the unit weight metric. The matrix of observations 
on the transformed independent variables is composed of the residuals obtained 
in the regression of each variable, except the first, on the elements preceed- 
ing it in the original vector. Table 3 contains the transformed regression 
Variables” (xX, + X, RWX, RW). Only a few diqits have been retained to make it 
easier to read the table. 


When we regress Y on (1, X - Xe RWX, RW) we obtain 


y POBNE ANS 12085 CXISeX) SEN S093* 107 *RWX - 0.015RW, 
[2eeag) ts) (0.044) (0.023) 
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where the estimated standard errors were computed for a stratified sample with 
unit weights using expression (14). If the reqression and standard errors are 


computed by ordinary least squares, we obtain 


eal plac 1.085(X - X) + 0.093 x 1072RWX - 0.015RW. 
C5357 ee COeat ty) (0.086 ) (0.034) 


The estimated standard error for the coefficient of RWX obtained by Taylor 
methods is about one half of that obtained by ordinary least squares methods. 
This can be explained by the data configuration. 

The first observation on RWX is much larger in absolute value than any other 
observation. Of the total sum of squares for RWX, 47 percent is due to this 
observation. The Taylor approximation to the variance uses the sample vari- 
mag ins e14)e-co estimate the variance of the statis- 
tic. The deviations from regression, denoted by v, are given in the _ last 


ance of deviates called d 


column of Table 3. The v value for observation one is among the _ smaller 
values. The mean square for the residuals is 421. The product (RWX) (v) for 
the first observation is -1113. This product is of the same order of 
magnitude as the product for observations 3, 33 and 36. . Therefore, while the 
first observation is responsible for about 67 percent of the sum of squares of 
RWX, it is responsible for only about 15 percent of the sum of squares of 
(RWX)(v) « This is because v2 for the first observation is less than one tenth 
of the average of the squares of the other observations. Furthermore, the 
squared deviation for the first observation is biased downward because the 
method of least squares will cause the estimated plane to pass close to an 
observation that is separated from the other observations. Thus, if all of 
the observations have the same error variance, the Taylor method will produce 
an estimate of the variance of the coefficient for RWX that is biased 
downward. 

Did the procedure underestimate the variance for this sample? We do not 
know. If we use the parametric procedure of ordinary least squares, we assign 
the pooled estimate of error variance to the separated observation. It is not 
possible to determine if this procedure is correct because our estimate of 


variance for the separated observation is a one degree of freedom estimator. 
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In this situation most people will feel more comfortable assumina that the 
variance for the separated point is the same as the variance of the other 
points rather than taking the small observed variance of the single point. 

In the nonparametric world a single observation contains little information 
about the variability of the population that qenerated the observation. 
Furthermore, an observation separated from other observations is essentially a 
single observation. In the full parametric world the separated observation is 
in the fold because the separated observation is specified to have been 
created by the same generating mechanism that created the other observations. 
For data of the type displayed in Table 3, the answer obtained by parametric 
methods rests very heavily on assumptions about the error variance. 

In the estimation of variances, one measure of the numerical size of the 
sample is the number of cluster degrees of freedom. Thus, for example, the 
estimated covariance matrix for a k-dimensional vector random variable is 


Singular unless 


(n, Nb) en Kw 


Tl so ft 


=a 
In setting approximate confidence intervals it seems reasonable to use Stu- 
dent's t distribution with degrees of freedom no greater than » (n, = 1 eee 
cause the variance of an estimated variance is a function of the fourth mo- 


ments of the population, estimated variances are notoriously unreliable. The 


dl 


coefficient of variation for the squares is 2” for the normal and considerably 
larger for many other common distributions. 

If the error variances in the strata are unequal or if unequal weights are 
applied to the estimates of different strata, the variance of the variance 
estimator can be considerably different from that suqgested by a simple calcu- 
lation of error deqrees of freedom. Table 4 has been constructed using the 
data configurations of Table 1 to illustrate these effects on the estimated 
variance. In the first column we assume that stratification is ineffective in 
that we assume each stratum variance is equal to the variance of the popula- 
tion. We assume the parent population to be normal so that we can give an 


explicit expression for the variance of the variance. In this situation stra- 


tification produces an estimated error variance for a mean with a variance 
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that is proportional to (26.6) + while a simple random sample produces ,a 
variance of the estimated variance that is proportional to Bere The effec- 
tive degrees of freedom for the stratified sample is slightly less than 27 
because of the unequal sample sizes within strata. If we use the sample 
weights of Table 1 and the usual stratified variance estimator, the variance 
of the eaeemaced variance is proportional to ealcanebs This large reduction is 
due to the large weight for the first stratum. If the variance in the first 
stratum is one half of the variance in other strata, then the effective de- 
grees of freedom for the variance estimator is 12.4. In the last column we 
give the effective degrees of freedom for the simple random sample if the 
variance of the simple random sample is twice that of the stratified sample. 
This illustrates the fact that stratification can reduce both the variance of 
the estimated mean and the variance of the estimated variance of the mean. 
While we are unable to specify the number of error deqrees of freedom 
required for our approximations, it is clear that we shall be uncomfortable 
with a small number of deqrees of freedom, particularly with unequal weights. 
The theory of Corollary 1 uses a linear approximation to the nonlinear 
function of the sample means to approximate the behavior of the nonlinear 
function. If this approximation is to perform well, the curvature of the 
function must be small relative to the standard error of the sample means. 


For example, if the function is quadratic 


g(Y) = aY a a2, 


the linear approximation is 
g(Y) Z ay + ay? + (a) + 2agu)(Y - W). 
The expected value of q(Y) is 


E{g(¥)} = ou + oly? + v{y}1. 


For the linear approximation to perform well we must have small v {y} and/or 
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small ag. 

In summary, to be comfortable with the use of large sample theory we 

require: 

‘its A reasonable number of observations in the sense that no observations 
are widely separated from the main clusters of observations. This is 
another way of saying that the Taylor deviates are such that the mean 
of the deviates is nearly normally distributed. 

Ze A reasonable number of effective error degrees of freedom for the 
estimator of variance. 

De The curvature of the nonlinear function of sample means to be small 


relative to the standard error of the sample means. 
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Table 1: Soybean Area Determined by Two Methods 


ee ee ee 


Soybean Hectares 


a tn es 
County Seqment Weight Interview (Y) Satellite (X) 
ae a ane eae a a aaa 
1 1 502 8.09 24.75 
1 2 106.03 98.10 
1 3 103.60 112.50 
Z 1 712 6.47 43.20 
2 2 63.82 gn.10 
3 1 188 43.50 61.65 
3 2 Tina 92.70 
3 S 42.49 74.25 
4 1 190 15.26 98.10 
a 2 76.49 99.45 
4 3 174.34 152.1 
5 1 134 95.67 57.60 
g 2 76.57 66.15 
5 3 93.48 91.80 
6 1 189 37.84 34.65 
6 2 te tech 2 97.65 
6 3 124.44 116.10 
7 1 172 144.15 136.35 
iu Z 103.60 99.45 
7 3 88.59 99.90 
fi 4 115:..58 123.30 
8 1 114 99.15 B5.50 
8 2 124.56 121.50 
8 3 110.88 77.40 
8 4 109.14 102.60 
B 5 143.66 195.05 
9 1 193 91.05 7515 
2 2 132255 85.95 
9 3 143.14 112.05 
9 4 104.13 81.90 
9 5 118.57 80.55 
10 1 93 102.59 117.90 
10 2 29.46 39.15 
10 3 69.28 72.00 
10 A 99.15 99.45 
10 “ 143.66 155.25 
10 6 94.49 85.50 
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Table 2: Estimated Standard Errors of Reqression Coefficients 


Calculated by Alternative Procedures 


Fstimated standard Error 


Procedure Bo By 
Ordinary least squares 10.747 OST 116 
Stratified; sample weights 8.52 Ui ds a 
Cluster; sample weights Tilcomeeal 0.0823 
Stratified; unit weights Delon 0.0963 


Cluster; unit weights . 1135255 OMO7 


Stratum Cluster 
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Table 3: Data for Transformed Regression Problem 


Weight ee x 107 2RWX RW 
ee 
502 -67 ~195 167 
502 7 25 336 
502 21 68 369 
212 -48 1 ; 
212 -11 4 24 
188 -30 10 a7 
188 1 5 7 
188 ty 8 a 
190 7 4 12 
190 8 4 13 
190 61 a3 38 
134 34 28 -53 
134 25 23 =51 
134 0 5 -47 
189 -57 13 -20 
189 6 4 11 
189 25 2 20 
172 45 -9 8 
172 8 3 A 
172 8 2 = 
172 32 5 3 
114 =6 10 | 
114 30 2? =h6 
114 =14 18 ~68 
114 1 5 267 
114 42 252 65 
193 -16 7 4 
193 ne 6 9 
193 21 3 22 
193 -10 6 7 
193 14 6 6 
114 26 Buf) ~90 
114 ~52 63 -84 
114 -19 26 hd 
114 8 =i -89 
114 64 65 -93 
114 ae 12 -88 
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Table 4: Efficiency of Estimated Variance under Alternative Assumptions 


Procedure 


Simple random sampling 


Strat. Sa., unit weights, equal var. 


Strat. Sa., unequal weights, 
equal var. 


Strat. Sa., unequal weights, 
04 = 0.50% 


Equivalent degrees of freedom 


VoRGha. ‘ct Wop 7 OcNae 
36 9 
26.6 26.6 
4.8 4.8 
13.9 13.9 
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Post ’81 Censal Redesign of the Canadian! 
Labour Force Survey 


M.P. SINGH, J.D. DREW, and G.H. CHOUDHRY! 


ABSTRACT 


ollowing each decennial population census, the Canadian Labour Force Survey (CLFS) has undergone 

sample redesign to reflect changes in population characteristics and to respond to changes in informa- 
ion needs. The current redesign program which culminated with introduction of a new sample at the 
eginning of 1985 included extensive research into improved sample design, data collection and estima- 
on methodologies, highlights of which are described. 


EY WORDS: Continuous survey; Multi-stage sample design; Stratification; Sample reallocation; 
Telephone interviewing; Raking ratio estimation. 


1. INTRODUCTION 


The Canadian Labour Force Survey (LFS), the largest monthly household survey conducted 
y Statistics Canada, has been redesigned in the past following each decennial census. As a part 
f 1981 post censal redesign, an intensive program of research as outlined in an earlier paper (Singh 
nd Drew 198la) was undertaken in the areas of sampling, estimation and data collection 
1ethodologies. As the reliability of labour market data at the national and provincial levels was 
f sufficiently high standard, the major emphases in this redesign were on improving the reliabili- 
y of subprovincial data and on making the survey more cost efficient. Towards the latter, the 
lain thrusts were on increased automation of various steps involved in sampling, greater use of 
ensus data in place of independently obtained information in updating the sample, and increas- 
d telephone interviewing as a regular feature of the survey. As for the improvement in the sub- 
rovincial data, alternative sampling and estimation methods were investigated leading to 
sfinements in the earlier methods, coupled with reallocation of the sample within provinces. 

This paper presents an overview of the findings of various theoretical and empirical investiga- 
ons and field tests undertaken during the redesign program. Sections 2 and 3 provide the 
ackground information on objectives and the old design, while Sections 4, 5 and 6 highlight 
1¢ findings of investigations leading to changes in sampling and data collection methodologies. 
ection 7 deals with estimation issues, and sample reallocations are discussed in Section 8. Im- 
lications of the changes made in the redesigned sample on other associated surveys are outlin- 
Jin Section 9, and finally in Section 10 the benefits from the major improvements in the 
sdesigned sample are briefly recounted, along with some mention of future research plans. 


2. OBJECTIVE SETTING 


A fundamental step in the redesign of a recurring survey is the re-establishment of survey 
bjectives. For the LFS, this involved reassessment not only of the survey’s principal role as 
provider of current labour market information, but also of its use as a central vehicle within 
‘atistics Canada for conducting household surveys (Singh and Drew 1981b). 
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At the early stages of the redesign program it was decided that while primary orientati 
towards the LFS should be maintained, efforts should also be made to enhance flexibility 
the vehicle for general applications. In this light, several changes are being adopted that v 
benefit not only the LFS but its associated surveys as well. Requirements for the labour mar] 
data are noted below, while highlights of the changes resulting in the improvements for associat 
surveys are given in Section 9. 

Objectives relating to provision of labour market data were established in consultation w 
the statistical focal points within each of Canada’s ten provinces, and with key federal u 
departments, through annual Federal/Provincial Conferences on Labour Statistics and bilate 
follow-ups. In general, these consultations revealed satisfaction with data reliability for p: 
vincial and national data, but a strong desire for improved subprovincial data. Specific dé 
reliability objectives adopted for the redesigned sample are as follows: 


(i) for Canada and each of the ten provinces, no reduction in current reliability for mont! 
estimates of level and estimates of month-to-month change in total employment a 
unemployment. 


(ii) for the 24 Census Metropolitan Areas as defined by the 1981 Census, monthly estima 
of unemployed with coefficients of variations (CV’s) of 20% or less. 


(iii) for 66 subprovincial Economic Regions agreed to in consultation with the province 
monthly estimates of unemployed with CV’s of 25% or less. 


(iv) for cities with population of 60,000 or more in Quebec and Ontario, and 25,000 or m« 
in the remaining provinces, quarterly estimates of unemployed with cv’s of 25% or le 


Attainment of these objectives necessitated a reallocation of sample within the provinces tl 
entailed a shift of sample from larger cities and Economic Regions to smaller ones. This, coup! 
with the desire to reduce the cost of the LFS as discussed in Section 8, created high expectations fre 
the research projects for identifying more cost efficient strategies for data collection and prodi 
tion of statistics from the LFS. In the following sections, these issues are addressed for the t' 
main parts of the survey namely, Self-Representing (SR) and Non Self-Representing (NSR) Are: 


3. OLD LFS DESIGN 


A complete description of the old LFS design is given by Platek and Singh (1976). Sali 
features are noted in this section to provide a context for discussions in later sections. 

The Self-Representing Units (SRUs) in the old design corresponded to those cities whic 
when designing the survey, were sufficiently large to yield an expected sample of 20 dwellin; 
the minimum thought acceptable for an interviewer. Minimum SRU sizes ranged from a popu 
tion of 10,000 in the Atlantic Region to 25,000 in Quebec and Ontario. 

Within large SRUs, deep geographic stratification was carried out by grouping togeth 
contiguous Census Tracts — geostatistical areas with populations in the 3,000 - 5,000 rans 
whose stability from one Census to the next makes them a convenient operational unit — witho 
any regard to optimality of characteristics. Primary sampling units, called clusters, correspo 
ding approximately to city blocks, were delineated on the basis of field counts obtained in 197 
A two stage sample of clusters and dwellings was selected following the Rao, Hartley and Cochr 
(1962) pps random group method. In addition to the area frame, an open-ended list frame 
apartments was maintained in the larger cities. 

A major advantage of the selection method for the area frame lies in its flexibility for chang 
in the sample size (Singh and Drew 1977), and for sample updating (Platek and Singh 197 
Drew, Choudhry, and Gray 1978). Sample updating is necessary in SRUs because over time t 
design counts used in the pps selection become out of date, leading to higher sampling varianc 
for survey estimates. Since sampling is done independently in each random group, a Keyfitz (1% 
sample update can be carried out, under which revised probabilities of selection bas 
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on recent dwelling counts can be incorporated, while maximizing retention of already selected 
units and avoiding overlap of selected dwellings between the pre- and post-update-samples. 
Regular updating of high growth SR areas occured from 1978 until the beginning of the redesign 
period in 1982, during which time almost half of the frame was updated. While the intensity 
of updating was not sufficient to reduce survey design effects to levels experienced during the 
first 4 years of the survey, it was sufficient to arrest further deterioration which had been 
averaging 7 - 8% per year for unemployed. 

The Non-Self Representing (NSR) units are the areas outside the SRUs comprised of rural 
areas and smaller urban centers. In NSR areas, stratification based on industry classifications 
was carried out within Economic Regions, subject to the restriction that strata should be con- 
tiguous land areas. Within strata, Primary Sampling Units (PSUs) were delineated such that 
each PSU represented its stratum to the extent possible with respect to the ratio of rural to 
urban population and important LF characteristics. While the rural portions of a PSU were 
comprised of collections of contiguous rural EAs, urban portions could not always be made 
contiguous to the rural due to the restrictions placed on maintaining the rural to urban popula- 
tion ratio. Frequently larger urban centres had to be shared amongst several PSUs within the 
stratum. 

At the time of the 1973 Redesign, two PSUs were selected per stratum using the randomized 
pps systematic method (Hartley and Rao 1962). In 1977, the LFS sample size was increased 
from 33,000 to 55,000 households per month, with the additional sample being allocated so 
as to improve data reliability at the province level. Thus the smaller provinces received larger 
proportionate sample size increases. The increase was achieved in NSR areas by selecting 1 - 4 
additional PSUs per stratum (Gray 1975). 

Within selected PSUs, urban and rural portions were sampled independently. In the urban 
portion of the selected PSU a two stage sample of clusters and dwellings was selected, whereas 
in the rural portion of the PSU a three stage design was used with secondaries (which are Cen- 
sus EAs or combinations), clusters (identifiable land areas having up to 20 dwellings) and dwell- 
ings as the sampling units. Except for the last stage, randomized pps systematic sampling was 
used in the selection. 


4. REDESIGN OF THE SELF REPRESENTING AREAS 


The criterion for cities to qualify as SRUs in the new design was raised to a minimum sample 

of 50 dwellings, since analysis of cost data indicated significantly higher unit costs for smaller 
SR assignments. The composition of the SR universe remained largely unaffected, however, 
due to the off-setting influences of the increase in the LFS sample size from 33,000 to 55,000 
households during the late 1970’s, and due to the reallocation of the redesigned sample. 
__ For reasons noted earlier, the basic design in the SR areas remained the same and the main 
thrust for these areas was to update the size measures without resorting to a costly independent 
field count as used in the last redesign. In order to exploit the data collected during the 1981 
‘census for this purpose different approaches were used in the block-faced cities (larger cities 
where data were available at block face level) and non-block-faced cities. The choices of up- 
‘dating method and sampling units by the two approaches are given below, whereas the two 
level stratification adopted for SR areas is given in Section 5. 


Block-Faced Cities 


The availability of Census data at the block face level in the built-up portions of the larger 
cities was the key factor in deciding to completely redesign the sample in these cities, which 
account for 74 of the SR frame. The redesign of these cities also provided the opportunity to 
‘introduce improved stratification as described in the next section. 
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For block-face portions of the cities, Census blocks were adopted as clusters (i.e., PSUs) 
Variance components under a two stage RHC random group design were studied for differen 
choices of clusters — Census Blocks or EAs — by simulating the LFS design using 1976 Censu: 
data for the SRUs of Halifax and Saskatoon (Choudhry, Drew, Lee 1984). Study results, fo: 
the case of up-to-date size measures, showed little difference between the EA and block in term: 
of sampling variances. Hence, the decision in favour of blocks was made on the basis of opera 
tional considerations. The blocks provided a ready made frame (with splitting or combinin; 
needed in only 5 - 10% of cases), permitting a highly automated design with very low redesigi 
costs. Importantly as well, data for future Censuses will be retrievable for the geostatisticall 
stable blocks (but not for EAs which as operational units change at each Census), permittin; 
low cost quinquennial updating of the sample. The built-up portions comprise 86% of these cities 

The EA was adopted in the non-block faced portions of the cities. Whereas the study results con 
sidered only the up-to-date case, it was felt that the EA being a larger unit than the block would b 
more robust to the highly clustered growth which can occur in the non-built-up portions of cities. Alsc 
adoption of the EA in these areas permitted very low redesign costs, since roughly 80% of thes¢ 
areas fell in Quebec and Ontario where due to the low sampling rates very little splitting was needed 

The variance study results, combined with cost results from the Time and Cost Study (Lemaitr 
1983), showed variances per unit cost to be quite flat in the range of 2 - 8 selected dwellings pei 
cluster. Hence it was decided to retain the density of 4 - 5 dwellings per cluster used in the olc 
design in strata where clusters were blocks, but to increase the density to 6 - 8 dwellings in the 
case of the EAs, due to their larger sizes. 


Non-Block-Faced Cities 


Since over 70% of the non-block-faced SRUs were either new or had changed boundaries, anc 
since most of those remaining had not been updated since the 1973 redesign, it was decided tc 
completely redesign these cities. Clusters were taken as individual or combined Census blocks ir 
the built-up portions of the cities, with the dwelling counts obtained directly from visitation record: 
and maps completed by Census enumerators. In the non-built-up portions, EAs or split EAs were 
taken as clusters, with field counts sometimes being required to do the splitting. 

The use of Census visitation records, while more costly than procedures followed in the block. 
faced areas, nevertheless resulted in significant cost savings over the procedure followed in the 
old design of obtaining independent field counts. 


5. STRATIFICATION 


5.1 Algorithm and Choice of Stratification Variables 


A modified version of a non-hierarchical algorithm due to Friedman and Rubin (1967) was 
adopted for stratification purposes in both SR and NSR areas, on the strength of findings reportec 
by Judkins and Singh (1981), and Kostanich, Judkins, Singh and Schautz (1981) who evaluated 
several stratification algorithms for use in the U.S. Bureau of the Census’ Current Population Survey. 
New features incorporated into the algorithm were a capacity to form geographically contiguous 
and/or compact strata, and the option to form either homogeneous clusters (i.e., strata) 01 
heterogeneous clusters (i.c., primary sampling units within NSR strata). A detailed descriptior 
of the method, and results of empirical evaluation studies are given by Foy, Bélanger, Drew and 
Joncas (1984). An overview is presented below. 

The algorithm starts with a random partitioning of units into a specified number of strata. 
An iteration consists of examining in turn each stratification unit and moving it to any stratum which 
will reduce a weighted multivariate within stratum sum of squares, while continuing to satisfy con- 
straints on strata sizes. The algorithm converges at a local optimum when moving any one of the units 
would increase the within strata sum of squares. Based on the findings of Judkins and Singh (1981), 
local optima were improved upon by the use of a moderately large number of random starts (i.e., 30). 
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For the contiguity option, a matrix is inputted specifying for each unit, all others contiguous 
to it. Contiguous initial strata respecting the size constraints are then built up from units chosen 
as random starting points. During the optimization step, an extra condition is imposed on transfer 
of units that contiguity be maintained. To achieve compactness, population centroids (longitude 
and latitude) are added as variables in the weighted sum of squares to be minimized. 

For both NSR and SR areas, a multivariate stratification has been carried out using 1981 
Census data for up to 17 stratification variables. Population variables include: total employed; 
employment income; persons with secondary education; population 15+, 15 - 24 and 55+; 
and labour force in agriculture, forestry-fishing, mining, manufacturing, construction, transport, 
and services. Dwelling related variables include: total dwellings, dwellings rented, one person 
households, and two persons households. 

Any industry variables accounting for less than 2% of the labour force of the area being 
stratified were dropped and other variables were scaled to be equally important in the optimiza- 
ion process. Unemployed was not included as a stratification variable due to its instability. 
Study findings showed that strata formed without unemployed when evaluated at the next Census 
were more efficient not only for other characteristics, but for unemployed as well. The inclusion 
of the neighborhood type variables, on the other hand, did result in improved efficiency for 
unemployed. 


5.2 Two Level Stratification in SR Areas 


In the larger SRUs with sample sizes of 300 or more households, two levels of stratification 
were adopted. Primary strata, with sample yields of 150 - 170 households from the area and 
apartment samples combined, are comprised of collections of geographically contiguous Census 
Tracts. As such, primary strata are designed to correspond to two interviewer assignments. Three 
fo four non-geographic secondary areal strata each yielding six or a multiple of six sampled 
clusters are formed within primary strata, with Census Tracts as stratification units, and with 
optimization based on the 1981 Census characteristics of non-apartment dwellers. 

Apartments are sampled separately, in the form of a pps systematic sample from an open- 
ended list, which generally comprises a single stratum for the entire SRU. Sorting of the apart- 
ments existing at the time of design by primary strata yielded an implicit geographic stratifica- 
ion to the apartment sample. 

In the smaller block-faced SRUs which warranted neither separate apartment sample nor 

zeographic primary strata, optimal non-geographic areal strata were formed directly. In the 
10n-block-faced cities, with considerably less scope for stratification, simple geographic strata 
vere opted for. 
_ The two levels of stratification in the larger SRUs had appeal on both operational and technical 
srounds. The relaxing of geographic constraints over those existing in the old design permitted 
sreater optimality to be achieved, while the retention of contiguity at a higher level will provide 
i suitable unit for sample updating purposes later in the decade, and will facilitate the planning 
of interviewer assignments. Also, in the old design, SR strata were likely to be covered entirely 
»y a single interviewer, and hence the variance estimates did not reflect the correlated response 
variance component of total variance. To the extent that within strata, interviewer assignments 
wre geographic and secondary strata are non-geographic, an interpenetration of strata and in- 
erviewer assignments will be achieved in the new design without incurring any additional data 
‘ollection costs, resulting in this component being better reflected in the variance estimates. 
| Table 1 presents study results for two SRUs — Ottawa and Quebec City — comparing effi- 
iencies of the geographic strata used in the old design with those of optimal two-level strata, 
ormed using 1971 Census data. Percent reductions in the first stage variance due to stratifica- 
‘on, calculated at the time of the 1981 Census, indicate largest improvements under the optimal 
tratification for income and rented dwellings. The only marginal gains for other characteristics 
acluding employed and unemployed point to the strength and robustness of the simple, but 
feep, geographic stratification in the old design. 


| 
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Table 1 


% Reduction in First Stage Variance Due to Stratification 
Old vs New Methods 


Variable Stratification Method Variable Stratification Method 
old new old new 
total employed 9.1 12.6 agriculture! 5.9 3.9 
employment income 18.1 30.4 forestry/fishing! Srl 2.4 
secondary education 39.4 42.1 mining! 4.8 3.0 
population 15+ 9.2 12.6 manufacturing 235) 23k 
population 15 - 24 12.9 76, construction 11.9 ley 
population 55+ 203 29.7 transport 4.2 6.4 
total dwellings 28.5 83x services 14.5 19.8 
dwellings rented 20.9 28.8 unemployed! WE 9.7 


1 person households 33-7) 38.4 
2 person households DS 29.6 


' characteristics not used in optimization for new method 


In the NSR areas, the same clustering algorithm was used within each Economic Region 1 
form either rural, or mixed urban and rural strata, depending on the design adopted, as discusse¢ 
in Section 6.3. Also the adaption of the clustering algorithm for use in PSU formation is describe 
in Section 6.5. 


6. DESIGN CONSIDERATIONS IN NSR AREAS 


6.1 Extension of Telephone Interviewing 


Telephone interviewing for months 2-6 in the sample was introduced during the early 1970 
in Self Representing Areas, primarily to reduce cost. However in NSR areas, all interviewin 
continued to be done in person due to concern over the high instance of party lines vis-a-v 
the confidentiality of the data being collected. Nevertheless, in recognition that not only in 
mediate cost benefits from telephoning were at issue, but so also were the longer term potenti 
benefits from the use of new technologies such as Random Digit Dialing and Computer Assiste 
Telephone Interviewing (CATI), it was decided to test the feasibility of extending telephon 
interviewing to NSR areas. 

A first field test was restricted to urban areas having over 80% private lines. The test we 
conducted on a portion of the actual LFS sample, with the principal objective of assessing th 
data quality implications of telephone interviewing. To facilitate this analysis, interviewé 
assignments were split between the telephone and personal procedure. 

This test ran from January 1982 to June 1983 with a gradual phase-in to ensure no advers 
impact on the ongoing survey. Principal findings were: lower non-response rates for the telephon 
sample (3.4% versus 4.3% for the control sample); a high instance of households with telephone 
(96% for all provinces but one); a low instance (1%) of households not agreeing to telephon 
interviewing; and no detectable differences in estimates for labour force characteristics. 

A second test carried out in the rural areas had comparable findings. Based on the positiv 
findings from both tests, the decision was taken to introduce telephone interviewing across th 
board in NSR areas during the remainder of 1983 and early 1984. 
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The decision to extend telephone interviewing had the following principal implications on 
he design of the NSR sample: 


(i) Increase in assignment sizes: In the old design, NSR assignment sizes averaged SO dwell- 
ings. Evidence that per unit costs were lower for larger assignments (Lemaitre 1984), 
and the reduction in travelling under telephoning, both supported increasing the design 
yield per NSR PSU to 55 - 60 dwellings. 


(ii) Level of assignment of rotation numbers: Unlike the old design, in the new design, all 
dwellings within secondaries will receive the same rotation number, which will cut down 
on the number of visits to the secondaries in month 2 - 6 in the sample. 


.2 Elimination of Stage of Sampling in Rural Areas 


In the old design, the rural sample within PSUs was selected in three stages: secondaries (Census 
numeration Areas), clusters, and dwellings. The clusters corresponded to identifiable land areas 
Ontaining up to 20 dwellings, which were delineated on the basis of field counts obtained 
vhenever a new secondary entered the sample. Within secondaries, generally 5 - 6 clusters, with 
}- 4 dwellings per cluster, were selected. 

The rural cluster stage was identified early in the redesign program as a possible candidate 
or elimination, on the grounds that (i) the sample variance would be reduced due to having 
yne less stage of samplings, and (ii) the lead time required to introduce new secondaries into 
he sample could be shortened from 13 months to 7 months. 

A field study was carried out on a sample of secondaries entering the LFS sample, in order to 
ssess the feasibility of maintaining good quality dwelling lists for entire rural EAs, and to examine 
osts under such a procedure, with positive results on both counts. The variance implications of 
liminating the cluster stage were also studied. Using 1971 Census data to simulate both the old 
ind alternative design, components of variance were obtained for the Horwitz-Thompson estimator 
vithout ratio estimation. The percent reduction in total variance under the alternative design was 
ound to range from 20 - 25% for major labour force characteristics (Choudhry, Lee, and Drew 
984). 

On the basis of these findings, an early decision was taken to eliminate the rural cluster stage 
yf sampling, and attention was turned to more global aspects of the NSR design. 


.3 Design with Urban/Rural Stratification 


The old design featured implicit urban/rural stratification. PSUs were formed to have approx- 
mately the same ratio of urban to rural population as the stratum, and within selected PSUs 
he urban and rural portions were sampled independently. A premise underlying the design was 
hat the PSU should correspond to an interviewer’s assignment. However, in practice this cor- 
espondence was weakened since in order to attain the desired urban/rural ratio, frequently the 
rban and rural portions of PSUs were not contiguous. 

_ As an alternative to the old design, D,; (with the rural cluster stage eliminated), a design, 
),, featuring explicit urban/rural stratification was studied. Like D,, the alternative design D, 
onsisted of 3 stages of sampling in both urban and rural areas. In urban strata, the stages were: 
'SUs (consisting of individual or nearby urban centers), clusters, and dwellings. In rural strata, 
he stages were: PSUs, (consisting of collections of nearby rural EAs), secondaries (EAs), and 
wellings. Under D,, both urban PSUs and rural PSUs were designed independently to yield 
amples corresponding to interviewer assignments. 

_ The two design alternatives were evaluated, from the point of view of variance and cost 
Choudhry, Drew, Lee 1984). In the variance study both designs were simulated for the case of 
\PSUs per stratum using design counts based on the 1971 Census, and study variables based 
n 1976 Census data. : 

"In terms of costs, a simple model was developed for D,, the old design under telephone inter- 
ewing, and components were estimated using results from a detailed Time and Cost Study 
vemaitre 1983). Relative costs for the travel components between designs D, and D, were estim- 
‘ed by means of a simulation study, in which average dispersion of the sample under the two 
esigns was obtained up to the second stage of sampling using the population centroids of the EAs. 
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Findings were that the design D, was 1.09 times as cost efficient as D,, and that from the 
combined perspective of cost and variance, D, outperformed D, with overall efficiencies of 
1.25 for employed and 1.05 for unemployed. 

Based on these findings, design D, was adopted in 70% of Economic Regions with suffi- 
cient urban and rural population to yield separate strata. In the remaining Economic Regions, 
with the exception of Prince Edward Island, design D, was adopted (see Section 6.6). 


6.4 Number of PSUs Selected Per Stratum 


In the LFS design, since the sample yield per PSU is fixed, the number of PSUs selected 
per stratum also determines the number of strata. In over two thirds of cases, the urban, rural 
or combined strata within ERs yielded only enough sample for 2 or 3 PSUs. Further stratifica- 
tion in these cases was ruled out on the grounds that there should be at least 2 PSUs per stratum 
to permit unbiased estimation of variance. 

The remaining ERs were stratified to the point of 2 or 3 PSU’s per stratum. Estimated first 
stage variance reductions over the situation under the old design of from 3 - 6 selected PSU’s 
per stratum were up to 14% for employed (Choudhry, Lee, and Drew 1984). The stratification 
was carried out using the clustering algorithm described in Section 5. 


6.5 Use of Clustering Algorithm in Formation of NSR PSUs 


In both the old and new LES, stratification is carried out prior to formation of NSR Primary 
Sampling Units. PSUs are delineated within the stratum to be as similar as possible with respect 
to stratification variables, while being as geographically compact as possible. PSU delineation 
which was carried out using the clustering algorithm noted earlier, required minimization of 
geographic and maximization of the non-geographic variables. 


6.6 Two Stage Design for Prince Edward Island 


For Canada’s smallest province, Prince Edward Island, sampling rates are 4% in order to 
produce monthly LF estimates with required levels of data reliability. In view of the high sampling 
rates, a less clustered design consisting of a two stage sample of EAs and dwellings, with deep 
geographic stratification was adopted. It was found to have marginally higher costs than D,, 
however from the overall perspective of cost and variance, it came out well ahead with efficiencies 
of 2.21 and 1.11 for employed and unemployed relative to D, (Choudhry, Lee, and Drew 1984). 


7. ESTIMATION 


7.1 Final Stage Ratio Estimation 


In the old LFS, a final stage ratio estimation was carried out by detailed province/age/sex 
cells. With the development within Statistics Canada of improved and more timely subprovin- 
cial population estimates (Verma, Basavarajappa, and Bender 1982), an intermediate ratio estima- 
tion step was studied in which survey estimates of population 15+ for subprovincial areas are 
ratio adjusted to external estimates prior to the usual final ratio estimation. Findings were that 
the procedure, while not impacting on the variances of provincial level data, resulted in variance 
reductions for sub-provincial areas ranging from close to 70% for employed to 7% for unemployed 
(Earwaker and Bélanger 1981). In practice a raking ratio procedure in which the two ratio estima‘ 
tion steps are iterated until both marginal controls are satisfied was adopted, beginning in 1983. 


7.2 Improved Estimates for Household and Family Units 


Paul and Lawes (1982) used LFS longitudinal data files, which link households over the six 
months in the sample, to demonstrate that non-response rates are higher amongst households 
with fewer members. For the old LFS, non-response adjustment consisted of re-weighting at 
local area levels. This was done without regard to household size, hence the resulting estimate 
of households and families by size had 1 - 3% biases. Another problem related to the inconsistency 
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yf family and individual based statistics (Macredie 1983). When demographic estimates of families 
yy size, currently under development by Statistics Canada’s Demography Division, become 
wailable, it is intended to incorporate them as an extra dimension in the final stage raking ratio 
stimation procedure, to address both problems. 

As an interim measure, the use of LFS longitudinal data is being studied as a mean to derive 
1ousehold size distributions based on both respondents and non-respondents, prior to the final 
aking ratio estimation (Ghangurde 1984). 


1.3 Small Area Estimation 


Demand for Labour Force estimates for small areas (domains) such as Federal Electoral 
Jistricts (FEDs) and Census Divisions (CDs), both of which number over 250 units across 
Sanada, has increased in recent years. Since it was not possible to respect the boundaries of 
uch areas in the design of the survey, various alternative small area estimation methodologies 
vere evaluated. A sample dependent estimator was proposed as a combination of post-stratified 
ind synthetic estimators, which relies entirely on the post-stratified estimator whenever the sample 
ize in the domain is sufficient according to certain criteria, and which otherwise introduces 
| synthetic component whose relative weight depends on the deficiency of the sample in the 
lomain. Based on study findings, it was recommended that the sample dependent approach 
ye developed as a means of providing annual or multi-year average estimates for areas such 
is FEDs and CDs (Drew, Singh, and Choudhry 1982). Implementation and further research 
ind developmental work is proceeding under Statistics Canada’s Small Area Data Program. 


.4 Variance Estimation 


The methodology for variance estimation for the redesigned sample will continue to be bas- 
d on Keyfitz’s (1957) method, although it will be further modified to the case of a two step 
inal stage ratio estimation, i.e., to a single iteration in the raking ratio estimation procedure. 
As subsequent iterations exert only a very small influence on estimates, they are being ignored 
n variance estimation. Some further refinements of the current variance estimation procedure 
ire under study, such as adopting clusters as replicates in SRUs, as opposed to the current practice 
f grouping clusters into two psuedo-replicates. 

It should be noted that variance estimators given by Rao, Hartley, and Cochran (1962), and 
y Rao (1975) were evaluated as alternatives to the current method in SRUs, where the RHC 
lesign is followed (Choudhry, Lee, and Sida 1984). The current method and the alternatives 
vere studied both with and without ratio adjustment. The current method without ratio adjust- 
nent was found to overestimate the variance for certain characteristics (e.g., 20% for employed), 
\owever with ratio estimation, biases were negligible. Estimated biases were also negligible for 
he alternatives. The principal advantage of the alternatives was that they were more stable. 
“he current method was retained however, due to its simplicity and also because of the com- 
lications in estimating variances of change or averages under the alternative methods. 


.5 Composite Estimation 


_ Inthe LFS, moderate to high month-to-month correlations exist for most characteristics due 
» the 5/6’th common sample. Different composite estimators were studied by Kumar and Lee 
(983), which take advantage of these correlations by use of data from previous samples to 
mprove the current month’s estimates. Their studies focussed on a class of AK composite 
stimators studied recently by Huang and Ernst (1981) and others in the context of the U.S. 
ureau of the Census’ Current Population Survey. 
| Findings under the assumption that the ratio estimator is unbiased, were that from the perspec- 
ve of mean square error, a compromise choice of the A and K weights yielded up to 5% gains 
ir monthly estimates of level for unemployed and employed, and from 5% - 16% gains for 
-_»rresponding month-to-month change estimates. A decision on implementation of composite 
‘timation was delayed pending further studies on the impact on composite estimators of any 
jianges in rotation group bias, stemming from modifications in non-response adjustment and 
itio estimation procedures, and pending closer examination of its operational implications. 
| 
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7.6 New Rounding and Release Policy 


In the old LFS estimates of level were rounded to thousands and released if greater than 
4 thousand. This policy was applied uniformly in all provinces for all estimates, with the intent 
that released data should have a coefficient of variation of 33.3% or less. 

More rigorous, provincially based rounding and release criteria were developed for the 
redesigned sample to satisfy the conditions that the CV of unrounded estimates should be 33.3% 
or less, and that the rounding error should not exceed 20% of the standard error of the unrounded 
estimate. Findings were that release criteria could be dropped to 2 - 3 thousand, for all pro- 
vinces except Quebec and Ontario, and that estimates for subprovincial areas should de round- 
ed to hundreds instead of thousands (Kumar 1982). 


8. SAMPLE REALLOCATION 


Specific data reliability objectives of the redesign having particular emphasis on better data 
at subprovincial level are noted in Section 2. In addition to the general improvements in the data 
reliability levels through the refinements in the methods and procedures, it became necessary to | 
consider reallocation of the sample within provinces to meet the reliability levels noted in objec- 
tives (ii), (iii) and (iv). Sample size increases were needed in 13 out of 66 Economic Regions, 6 
out of 24 CMAs and 27 out of 42 non-CMA cities. An average 28% reduction in the CV’s for | 
unemployed was obtained for these cases. In addition, for the 30 ERs with old CV’s in the range 
of 15 - 25%, the reallocations achieved an average 12% reduction in CV’s. As a rule of thumb, 
under the redesigned sample, monthly data for ERs and CMAs and quarterly data for other cities 
will be based on minimum monthly sample sizes of 300 and 120 households per month respectively. | 

It is worth noting that the redesign objectives did not directly consider two important uses 
of LFS data by federal government departments. These are the use of 3 month moving average | 
unemployment rates for subprovincial Unemployment Insurance (UI) Regions in determining 
the regionally variable number of weeks worked to qualify for UI benefits, and the use of 3 
year average data for 180 - 200 areas consisting of individual or combined Census Divisions, 
for use in allocating federal funds to assist new industrial initiatives. However, the re-distributions 
of the sample will indirectly benefit both of these applications. 

In determining sample size requirements to meet the objectives, average unemployment rates 
for the period 1980 - 82 were used, in view of medium term forecasts for sustained high 
unemployment during the 80’s. 

A general implication of these reallocations was the movement of a significant proportion 
of the sample from larger CMA’s and Economic Regions to smaller ones. This had an adverse . 
impact on the provincial and national estimates due to the departure from the usual propor- 
tional allocations. This decrease in reliability at higher levels was however more than compen- 
sated by the general increase in the reliability achieved through the structural improvements 
in the methods and procedures as a result of research investigations. 

A study was also carried out using the cost-variance model suggested by Fellegi, Platek and > 
Gray (1967) to arrive at optimum sampling rates in the NSR and SR areas. This resulted in 
a shift of sample from NSR to SR areas . This was most pronounced in Quebec and Ontario 
where the proportion of SR sample increased from .60 to .72, as compared with .78 of the; 
frame, yielding gains for provincial estimates of unemployed equivalent to a 5% variance reduc- 
tion, assuming a fixed sample size. In addition, this optimization helped in achieving objectives | 
(ii) and (iv), and it benefitted the Survey of Consumer Finances and Rent Survey. . 

It is worth noting that the structural improvements in the design achieved through factors. 
such as improved stratification, reduction in a stage of sampling in the NSR areas, incorpora- 
tion of subprovincial controls in the estimation procedures, and more refined sample alloca- if 
tions resulted in better than current reliability levels for national and provincial estimates while. | 
meeting the objectives for the subprovincial data. This opened up the possibility of reducing | | 
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the overall sample of the LFS in order to release funds for the collection of data on certain 
_ other socio economic issues from time to time. The size of the redesigned sample was therefore 
fixed as 51,500 households per month down from 55,000. This overall reduction of 6 - 7% was 
achieved through a uniform reduction in all provinces with the exception of Prince Edward Isiand. 
In addition, per unit data collection costs will be reduced due to increased telephone interviewing. 


9. IMPLICATION OF CHANGES ON LFS ASSOCIATED SURVEYS 


Most of the household surveys conducted by Statistics Canada take advantage of the invest- 
‘ment the LFS represents in terms of sample frame and design, data collection vehicle and pro- 
cessing systems to obtain data more quickly, at less cost and greater reliability than would be 
‘possible through independent surveys. The design and operations of these surveys are integrated 
with those of the LFS to varying degrees. 

Most common are supplementary surveys consisting of additional questions to LFS respon- 
dent, which incur only incremental costs of the extra questions. Surveys which due to the sen- 
sitivity of the subject matter due to the length of the questionnaire, might impact on the LFS 
‘are not done as supplements. Typically such surveys have been conducted by LFS interviewers 
on a separate sample of households selected in the same areas as the LFS. Less closely integrated 
with the LFS are surveys which select different areas from the LFS, but which benefit from use 
of the LFS sample design and from control of overlap with the LFS sample. 

As noted earlier the main orientation of the redesign program was towards the LFS, but efforts 
were also made to enhance flexibility of the vehicle for general applications. In this light changes 
being adopted for the LFS that will benefit its associated surveys are briefly highlighted below: 

The sample reallocation resulting in a shift of sample from NSR to SR areas will be more 
robust for general applications and in particular will improve estimation of income and rent 
changes from the SCF and Rent Survey. Also, the adoption of a 300 household minimum sam- 
‘ole size for CMAs will benefit these surveys for which CMA estimates are produced. 

The general multi-variate stratification using 15 variables adopted (in both NSR and SR areas) 
‘vill also represent an improvement for non-LFS applications over the current industry specific 
or simple geographic stratification. 
Three changes will specifically benefit surveys using different sets of households: (i) the elimina- 
ion of a stage of sampling in rural areas will considerably shorten the lead time to 7 months 
| ‘rom 13 months in the old design, (ii) PSUs will be more geographically compact due to the 
_idoption of explicit rural and urban strata, which will benefit smaller surveys where greater cor- 
| ‘espondence between PSUs and interviewers assignments is needed, and (iii) the flexibility in- 
_ roduced through the refinements in the sample stabilization program will allow selection of 
~ ubsamples of virtually any size at the national, provincial or subprovincial levels for surveys 
t ising the LFS vehicle. 
_ Finally the modification in the method of estimation introduced in the redesign, in the use 
1 if a household size distribution in the ratio adjustment as an interim measure (with eventual 
7 acorporation of demographic estimates of families by size as an additional dimension in the 
_ inal staged raking ratio estimation procedure) will improve the consistency amongst family 
~ \nd individual related labour market statistics, and will improve family statistics on expenditure 
ind income. 


a ——————— 
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a 10. SUMMARY OF CHANGES AND FUTURE RESEARCH PLANS 


' Most of the research investigations for the post-1981 Censal redesign of the LFS have been 
ompleted, with the implementation of the new sampling design underway and certain aspects 
f research in estimation methodology still to continue. It is worth noting that many of the 
| 
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investigations carried out during this program have confirmed the soundness of the past methods 
and procedures used in the LFS such as those of sampling two PSUs per stratum in the NSR 
areas, use of the RHC method and existing density factor (4 - 5 households per cluster) in the 
SR areas and continuation of the six month rotation pattern. However, several investigations 
have as well lead to improvements both in the redesign process and the new survey design; 
primarily due to change in emphasis on the data reliability objectives (as noted in Section 2), 
availability of better and easily accessible information and technological advances. 

Improvements in the redesign process included the use of 1981 Census data in place of in- 
dependently obtained field counts for updating the SR sample, the reduction of the clustering 
operation, and automation of stratification and PSU formation. Also cost savings will result 
from phasing-in much of the redesigned sample in an ‘‘on-line’’ fashion. Under this approach, 
the new sample will be introduced and the old sample will be dropped one rotation group at 
a time over a 6 month period, as compared with the traditional method of keeping the old sample 
at full strength for a 3 - 4 month period while building up the new sample (Mayda, Drew and 
Lindeyer 1984). Process cost savings as compared with the previous redesign are estimated at 
$1.8 million (in 1983/84 dollars). 

Principal improvements in the cost efficiency of the LFS survey design include the extension 
of telephone interviewing to months 2 - 6 in the sample in NSR areas, the adoption of an NSR 
design featuring explicit urban/rural stratification, elimination of a stage of sampling in rural 
areas, the general purpose stratification in both SR and NSR areas, the use of subprovincial 
population controls in the estimation procedure, and more refined sample allocation procedures. 


These improvements were sufficient to permit gains in the reliability of subprovincial data, while | 


retaining the status quo for provincial level reliabilities, and while decreasing the overall sam- 
ple size by 6 - 7%. Reliability gains averaged 14% for coefficients of variation of unemployed 
for the half of the Economic Regions and CMAs with poorest reliabilities under the old design, 
with for the most part, little or no change in remaining areas. Subprovincial gains for estimates 
of employed will be even greater. On the cost side, the sample size decrease, coupled with re- 
duced costs due to the extension of telephone interviewing will result in estimated cost savings 
of $.7 million per year (1983/84 dollars). 

Following the completion of the sample redesign a principal focus of design related research 
and development for the LFS in coming years will be on investigation of a dual frame 
methodology, underwhich a portion of the sample would be converted to a telephone frame, 
using Random Digit Dialing (RDD) techniques. A multi-year program of RDD testing including 
research into implications of higher non-response to the RDD telephone sample, of research 
into dual frame estimation methodologies and of study of centralization versus decentraliza- 
tion of telephone interviewing is currently in the planning stages, as part of a newly established 
telephone survey development program (Hofmann, Drew, Catlin and Mayda 1984). Another 
design related initiative will be aimed at developing cost efficient means of intercensally up- 
dating the area sample in SR areas. 
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Complex Designs: Some Canadian Experiences! 
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ABSTRACT 


Goodness of fit tests, tests for independence in a two-way contingency table, log-linear models and logistic 
regression models are investigated in the context of samples which are obtained from complex survey designs. 
‘Suggested approximations to the null distributions are reviewed and some examples from the Canada Health 
Survey and Canadian Labour Force Survey are given. Software implementation for using these methods 
is briefly discussed. 


KEYWORDS: ,’ statistic; Wald Statistics; Goodness of fit; Independence in two-way tables; Log-linear 
and logistic regression model. 
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; | 1. INTRODUCTION 


A sketch of the historical development of modern categorical data analysis has been given 
f n the excellent review paper by Imrey, Koch and Stokes (1981). These techniques, applied in 
: he context of random samples derived as independent selections from a common distribution 
_ ‘unction, are not directly applicable to survey samples collected using complex survey designs. 
‘| Koch ef al (1975), Shuster and Downing (1976), developed asymptotically valid methods, 
_ dased on the Wald statistic that take the survey design into account, but requiring access to 
] he micro-data file or at least the full estimated covariance matrix of cell estimates. Cohen (1976) 
_ ind Altham (1976) proposed a simple model for clustering and showed that the generalized 
b Wald statistic for goodness of fit is a multiple of x*, when the model holds. Brier (1978) con- 
_ idered a similar model, but studied general hypotheses on cell probabilities, and proved that 
Bh multiple of the corresponding Pearson statistic is asymptotically distributed as a x? random 
_ variable, when the model holds. Fellegi (1980) deflated the x? using a correction factor based 
on the mean of the estimated design effects. Fay (1985) developed jackknife x? and G? 
tatistics, also taking the design into account, but requiring the cell estimated at the primary 
ampling unit level. Rao and Scott (1981) developed a correction to x? (or G*) based on the 
jatterthwaite to approximation to the asymptotic distribution of x’, requiring the full estimated 
‘ovariance matrix. 

__ In this paper, we discuss the problems of fitting models and testing hypotheses with categorical 
er resulting from complex designs. For data collected using complex designs, some adjustments 


» the classical methods described by Imrey, Koch and Stokes (1981) are necessary in order to make 
alid inferences. If the published tables are provided along with the cell and marginal design 


‘This paper is a revised and expanded version of that presented at the Seminar on Recent Developments in the 
Analysis of Large Scale Data Sets sponsored by Statistical Office of European Communities, November 16-18, 1983, 
‘Luxemburg. 
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M.A Hidiroglou, Business Survey Methods Division, S. Kumar, Census & Household Survey Methods Division, 
Statistics Canada, Tunney’s Pasture, Ottawa, Ontario, Canada, KIA OT6, and J.N.K. Rao, Department of 
\Mathematics and Statistics, Carleton University, Ottawa, Ontario, CANADA. 
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effects, some of the approximations to the null distributions of our test statistics can be obtained, 
without having access to the complete micro-data file. On the other hand, for applications where 
the complete micro-data file is available, alternative approaches will be described. 

For illustrative purposes, Section 2 begins with the standard goodness of fit problem. This 
discussion is then extended in Section 3 to tests of independence in a two-way contingency table. 
This leads to a general discussion of log-linear models in Section 4. Logistic regression models 
are described in Section 5. In Section 6 we summarize the existing situation with respect to soft- 
ware development for these methods at Statistics Canada. In Section 7, we discuss the ap- 
propriateness of these methods. Numerical examples are taken primarily from the Canada Health 
Survey. An application from the Canadian Labour Survey is given in Section 5. 


2. GOODNESS OF FIT 


2.1 Multinomial Sampling 


Suppose we select n independent and identically distributed observations Lee :2)5 Yi 
a discrete distribution with k categories, where Pr(Y = i) = 7,52 ‘_, 1, = 1. We observe the 
random vector n = (n,, ..., m,_,)’, which has a multinomial distribution. Our estimate of 
TH oe es (oe) is given by p = n/n. This estimate is unbiased and has covariance matrix 
given by V{p} = (D. — aa')/n = P/n, where D = diag {m,, ..., 1,_,}. Note that P= 
D-' + (117/x,). Asymptotically, n’“(p — 2) > N(Q, P). For a given 7, the goodness of fit 
problem is to test the hypothesis. 


A eto ir. 
against the alternative 
Aya Fo. (2) 


Letting P represent P evaluated at z,, the Wald statistic for this test is 
W, = np-t,)'P,' (p-,) 
K 
ae nh {(p;- T,,) tf} ’ 


which is the familiar Pearson chisquare test. Under H, this is asymptotically ey: The 
likelihood ratio test for this problem is given by 


k 
LR, = 2n L D, log(p,/7,,)- 


Since 2p, log(p,/z,,) is asymptotically equivalent to 2(p,—7,,) + (p,—7,,)°/ m,, under H_,, we see 
that the likelihood ratio test is asymptotically equivalent to the Pearson chisquare statistic 
under H,. 

Another possible test for this hypothesis is derived by defining the vector of logs, w, = log 
m, and ~ = log p. Now under the null hypotheses @—y, is asymptotically equivalent to 


~oO 


D.\(p— 1,). Therefore, n*(a—p,) > N@, D'— 117) under H, and the Wald statistic is 


W, = (a-u,)" [D,. + G,x/2,.) )G-u) 


II 


k 4 p | 
L a, i) ’ 


i=1 


where p,, = log m,, and fi, = log p,. | 
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_This approximation is obtained by noting that under H, 


MiB Veo) Pea Rig 


— (p-1,)"l - -x,)"2, - 


Note that W, is also asymptotically equivalent to the Pearson chisquare test under ei. 
2.2 Other Sampling Schemes 


These results for W,, W, and LR, are well-known. The question of interest to us here is the 
implication of the more general assumption that n”(p — 1) > MO, V), where Vis not necessari- 
ly equal to P. Here p is a survey estimate of and may depend on sampling weights and other 

adjustment factors. This situation often arises in sampling under a complex sample design. We 
assume that Vis a consistent estimate of V. There are two approaches which we shall consider 


here. The first is to construct the appropriate Wald statistic for the given sample design. This 
would be 


where the rank of Vis k—1 so that W, is asymptotically x;_, under 1 Oe 

An alternative approach would be to use W,, W, or LR, directly as a test statistic. Now 
‘rom multivariate normal theory, we know that the distribution of n(p—1,)’P, \(p — x) is that 
of © 6, Z?, where {Z;} are independent x; random variable and 6 = (6, ..., 5,_,)’ are the 
sigenvalues of P,'V; see Johnson and Kotz (1970, pg. 150). This result was shown by Rao and 
scott (1981), who call the 6s generalized design effects. We note that for k = 2, we have 
i no*/{n (1 — m,)}, where o°* = V{p}. This is the usual design effect for p under H.. 


..3 Approximations 


Now, in general, the distribution function for linear combinations of x; random variables 
s cumbersome, although their moments are easily obtained. Rao and Scott (1981) have sug- 
‘ested two approximations to obtain the significance levels. The first is to approximate the 
listribution as being proportional to a x;,_, random variable, the proportionality constant 
veing determined by equating the mean of the approximating distribution to that of the theoretical 
‘istribution. This results in the approximation 


= k=l 
sr, O22 st y b/(K—- 1) } x7, (2.2) 
jow, 
b 6. = tr(P,'V) 
| k 
| = Xu Y,/ 10 
k 
= Pdi Khia) 


‘hich depends only on the cell design effects {d,}, where v,, is the i— th diagonal element of 
“and d, = v,,/[m,,(1—7,,)]. This approximation is particularly convenient when the full 
variance matrix is not known, but the cell design effects are given. This is often the case for 


fficial published data. 
| 
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Table 1 


Age Distribution Among Those Consuming 1-6 Drinks Per Week. 
Census Age Distribution for Canada (1978-9) 


Age 
Census 15-19 20-24 25-34 35-44 45-54 55-64 65+ Total 
Distribution 1338) m127 218 mS) .140 OS) ai 15 1.000 


Distribution of 
those consuming eI, .150 .264 .175 .148 .093 .053 1.000 


1-6 drinks/week 
Design Effect 1.4 1 Pp) i) 0.6 1.1 1.0 : 


Example 1 


For the Canada Health Survey (1978-9), a stratified multi-stage household survey, data was 
derived for the age distribution among those consuming one to six drinks per week, based on 
a sample of 5,204 persons, aged 15 years and over. A description of the survey may be found 
in ‘‘The Health of Canadians’’ (Statistics Canada Catalogue No. 82-538). 

The data, taken from Hidiroglou and Rao (1981), are presented in Table 1. The raw value 
for W, is 298. This is reduced to 248 by taking the approximation given by (2.2). For these 
data, the post-stratification adjustments for age and sex lead to small design effects. 

A second approximation to the distribution of L 6, Zt, suggested by Rao and Scott (1981), 
is the Satterthwaite (1946) approximation: L 6, Z? ~ ax?. To obtain a and », it is necessary 


to compute 


x6; = {(P'V¥} 
Ale nes 
= 7 2, Vik ( eBie)s 


However, this depends on all the terms of the matrix V. The important point, though, is that 
some adjustment to the multinomial test statistic is necessary to obtain the appropriate 
significance level. | 

An alternative approximation, suggested by Fellegi (1980), is to divide the statistic 
n(p—,)’ P,'\(p—7,) by the average design effect, d , instead of the weighted average given 
in (2.2). The effect of this on the data in Table 1 is that the adjusted chisquare value is 243, 
which is comparable to Rao and Scott’s (1981) approximation. 


3. TESTS OF INDEPENDENCE IN A TWO-WAY TABLE 


3.1 Multinomial Sampling 


We now suppose that the categories of the multinomial distribution can be cross-classified 
into an rxc table, where for the bivariate observation (Y,, Y,) we have Pr(Y, = i, Y, = j/) = ™ 
iT Cc a Pe Cc ed r = 
LES ie wats eaveb: Mee denote m,, = L)_, 7, and Ty = Uj ey We denote R= (My -05 
ial 0 ON) Ty carers 8) 5) Tp = (a4) recy Rg ’ To = rats at) Rae Ay) 9 Pp = Dr, — TBR 
=D ’. We observe the random vector n from the multinomial distribution, where 


@ SEG; 
EA) -—=“nre We let.p =n peli UP and p,,; = & Dy. 


iP? 


| 
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_ We wish to test the hypothesis of independence 
My area emuee -O LOL oN ice Pad dt gjrecucst, 


gainst the alternative 


Hi: Ree hy 0 for some (i, /). 


_ If we construct h, aNd ae a oe , for 1 < i< r—J and 1< j< c-—l, then under multinomial 
ampling under H.,, "the A eripiote. covariance matrix forh = (h,,, ..., igo 1 ee 
ye, ® PB. where ®@ denotes the direct matrix product operation. Hence, the Wald statistic 
Inder H, peeonies 


II 


W h'(P-! @ P.')h 


4 EC 


r 


] 
M 
M1 

S 


2 
y ¥ P;,P,;) M(D;,P .;)s 


i=1 j=1 


e familiar chisquare test with (r—1) (c—1) degrees of freedom. 
_ Another test, which is asymptotically equivalent to W, under H.,, is the likelihood ratio test 
ven by 


LR, = 2n[ 2% 2 p, log p, — x fey 0545 OP oe D., log p,, |: 


An alternative approach for this problem, which is a special case of the methods described 
1 /Grizzle, Starmer and Koch (1969) is to consider a Wald statistic based on 


Bie, = log p, — log p,, “logy; forl < i<'r—t,and] < j< ¢€—1}. 


Beeviipronc covariance matnx for f = ys <n<4 Jy gs oo Jee a ee 


D..' — 11"). Therefore the Wald statistic becomes 


— 
el 


Th Trip ike 
W. = f7|G,, + == )@ (b, += VF 


ie } r+ oni 


4 : : F : 
tw under H, we note that i; is asymptotically equivalent to 


1) 


| Ee ke 
= SB: 
’ W3= 2X p pet ee 


It should be noted that under H., the statistics W,, LR, and W, are all asymptotically 


2 2 
> ; 5 Pi tots as y (P;, i m,,)° 2 > 57 7.) 
} i= Ate 1 T™ t=1 1 j=l T 

| it +7 1+ TS 


(3.1) 


's result will prove useful in Section 3.3. 
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3.2 Other Sampling Schemes 


Now, relaxing the assumption that n is multinomial, we assume instead that n“(p— 7) > 
N(Q, V) where p is a survey estimate which may depend on sampling weights and other ad- 
justment factors. For this case, Shuster and Downing (1976) and Fellegi (1980) suggest that 
we construct the Wald statistic based on th; oh — Dee If we let J, be the (a—1) x @ 
matrix given by 


y 


kon) 


(3.2) 


and let A = (U = apl")e @,. — wal’) = Gh ehmed) 


then the Wald statistics is 


which under H, is asymptotically ene ; 

Alternatively, we could construct a Wald statistic based on { J, = log p, - log Dae 
log Poa This is a special case of the log-linear model approach to be discussed in Section 4. 
We define (r — 1) x rand (c —1) xX c matrices as follows: | 


Now, analogously to the goodness of fit problem in Section 2, Rao and Scott (1981) have 
considered null distributions of the test statistics based on W,, LR, and W,, which are all 
asymptotically equivalent to the null distribution of (3.1). We see, therefore, that the null distribu- 
tion is the same as © {7,6 , Z*, where {Z?} are independent x; and the 6,’s are the eigen- 
values of 


(Prep Baa) LV Hig): 


Cowan and Binder (1978) investigated the properties of the eigenvalue from a simpl< 
two-stage self-weighting design for a 2x2 table. They found that the eigenvalue increases 
as the degree of independence of the cell proportions within the primary sampling units 
decreased. 
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3.3 Approximations 


An approximation for the distribution of L Os 25 is 


Sp Zim on 


(7-1) (c-1) 


2 
X(r=1) (c= 1) ? 


as in (2.2). Since the statistic is asymptotically equivalent to(3.1) under H,, by computing the 
mean of (3.1) we obtain 
: 


BS = eed, — 74%, 5.2,49.0 -.m,) — © dPU-nz,,), 
i= j=1 


where d,, is the cell design effect; d‘ and d are the row and column margin design effects, 
“espectively. This particularly simple expression was obtained by Rao and Scott (1983). Fellegi 
1980) suggested an alternative approximation as: 


r Cc 
a Zz 
CG Beary 


c-l1) 


j Example Z 


_ InTable 2, we give a 4 x 2 table from the Canada Health Survey, which cross-classifies drug 

_ ise (four categories; 0, 1, 2, 3+ drug classes in a 2-day period) and sex (male, female). Here 
1 = 31,668. 

_ The raw value for W,, is 774. Rao and Scott’s (1981) adjustment reduces this to 437. Fellegi’s 

_ 1980) adjustment reduces this to 327. The Wald statistics, W,, is 538. Hidiroglou and Rao 

_ 1981) found that the Rao and Scott (1981) approximation performs quite well relative to the 

_ atterthwaite (1946) approximation which is based on the complete covariance matrix. 


LOG-LINEAR MODELS 


_ .1 Multinomial Sampling 


We now extend the results of the previous section to more general cross-classifications of 
te multinomial distribution. The standard results for these models are given in Bishop, Fienberg 


| Table 2 
Variety of Drugs Taken by Sex for Canada (1978-79) 


Number of Drug Varieties 


Sex 0 1 2 3+ Total 
_| Male Proportion 0.293 0.134 0.048 0.021 0.496 
Design Effect 1.56 3.37 1.15 1.38 0.00* 
Female Proportion 0.228 0.159 0.072 0.045 0.504 
a Design Effect 3.59 3.13 2.86 1.96 0.00* 
| Total Proportion 0.521 0.293 0.120 0.066 1.000 
At Design Effect 6.03 6.46 1.65 2.57 


_ * Because of age-sex post-stratification, these design effects are zero. 
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and Holland (1975) and Fienberg (1980). We have z = (TE aD is a vector of cell propor- 


tions; U*_, t, = 1. We observe n = (n,,...,n,)", the counts in each cell from a random sam- 
ple, so that m has a multinomial distribution (© n, = n). We let p = n/n and define 


The log-linear model assumes that for a parameter vector 6 = (0, ..., ae we have 


u(@) = u@)l + X8, 


where X is a known kx¢ matrix of full rank and X71 = 0. Note that ¢< k-1. If 
t = k—1, we have the saturated model. 
The maximum likelihood estimate for @ is given by solving 


Xp - #) =0, (4.1) 


where # = zm (6). Now, asymptotically we have 


t- 17 = PX@-8), 
where P = D — aa’. From (4.1), we then obtain 
QF 0 SCE eee ea) 


and 
a — w= PX(X'PX)'X' (o-2). 


Since n”? (p— 1) > NO, P) we obtain 
n® (6 -8) > N[0,(X7PX)"] 
ni? (x—x) > NO, PX(X TEX) XT PI 
Suppose now that the linear expression X@ can be decomposed as X,0, + X,@, where 4] 


and X, are full rank, X, isk x r, X,isk x s,6,isr x land @,iss x J, wherer + s =. 
We consider the problem of testing 


against the alternative 


t 


We use @,, @,, 7, etc. to denote the estimates under the full model H,. Alternatively, we 
let @ ,, z, to denote estimates under H, . 
Now, 


n'® (9, — 6,)> N[0, (XIPX,)~}] | 


where 


Xx, = 


a 
| 
de 
a 
es 
its) 
ine] 
VY 
\ 
x 
=a 
ae) 
’ 
nN 
as 
a 
a 
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so that the Wald statistic is 


Under H.,, this is asymptotically equivalent to the Pearson chisquare statistic 


n(@ — #)'D'G& - 0), 
or the likelihood ratio test 
LR =827 3k plog(t / ft). 
_ Under H., these statistics are asymptotically x? . 


4.2 Other Sampling Schemes 


We still assume that the cell proportions, z, satisfy » = log 7 -u(@ i Oy) Loren a 
X,@ , but we now have n”(p — 1) > N(O, V), where p is a survey estimate. 

Rao and Scott (1983) suggest the following Wald statistic for testing @, = Q. We let Cbe 
any k xX s matrix with C’X, = 0, C’l = 0 and C’X, nonsingular. For example if 
X{ X, = 0 then C = X, is convenient. Now the hypothesis is equivalent to C7 = 0. We 
have 


= CAC Pe Oo), 
where a is obtained from (4.1), based on the survey estimate, p. 
We therefore have the Wald statistics 
W, = np’C[CTX(X™PX) '(XTVX)(X™PX) 'X'C]'C'R. 


_ Similar results were also given in Binder (1983). If under H,, the model is saturated 
 (r+s = k—J), then p = 7 and we obtain 
Wo = me CIC DVD IC) “Cin. 

Rao and Scott (1984) show that if we use P instead of V in W, then these are asymptotical- 
ly equivalent to the likelihood ratio or Pearson x’ test statistics. They also show that the 
_ likelihood ratio test statistics is distributed as £}_, 6.Z? under H,, where {Z*} are indepen- 
_ dent x; and {6,} are the eigenvalues of 


| 
| (XTPX,)-'(X7V¥,), (4.3) 


| for X, defined in (4.2). 
| 


_ |4.3 Approximations 
| 


Z As before, we approximate the null distribution 


z pag aL 

Ba 2? = (=D) x. 
__ This involves computing the trace of (4.3). Rao and Scott (1984) show that if the model ad- 
‘mits explicit solutions for both # and #, then the approximation depends on the matrix Von- 
ly through cell design effects and marginal design effects. This observation is particularly con- 
venient when ony the estimated design effects for the cell proportions and margins are available, 
as is often the case for published tables. 
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Example 3 


Hidiroglou and Rao (1983) considered all direct estimates from the three-way table: Drug 
use (5 categories: 0, 1, 2, 3, 4+ drug classes in a 2 day period) x Age (4 categories; 0-14, 15-44, 
45-64, 65+) x Sex (male, female), taken from the Canada health Survey. We give the results 
for testing whether Age and Sex are independent in each drug category (n = 31,668). This is 
equivalent to the hypothesis 


Using Bishop, Fienberg and Holland’s (1975) notation, where log Ti = U+ Uy + UR 


+ Usyy + Ur + Usa + Yosye + Y“raxqjm» the null hypothesis is equivalent to 
i: Uns = Upsey = 0 for all (i, j, k). 


The raw chisquare value is 23 based on 15 degrees of freedom. The average eigenvalue is 
1.39, so that the approximation reduces the chisquare value to 16. Whereas the unadjusted chis- 
quare value would lead the analyst to reject the hypothesis at the 10% level, the approximation 
indicates that h, cannot be rejected even at the 30% level. 


5. LOGISTIC REGRESSION MODELS 


5.1 Multinomial Sampling 


We now consider a logistic regression model for the conditional distribution of a binary 
response variable y given the vector x of independent variables. In particular, this conditional 
distribution is 


POs wey (ttle 


where y, € {0, 1}. 
For the logistic regression model, we have 


log | see aha CL 
bas 1(X,) 


where @ is an unknown vector of parameters. 

We note that if x, is a categorical vector of 0’s and 1’s, this is a special case of a log-linear 
model as described in Section 4. Here we allow x, to be arbitrary. The extension to the case 
of k-categories for the y-variable is straight-forward, it is also possible to generalize the model to 


m(Xx,) 
log 


i= 5| = Sx; @), 


for a known function f(-), but we do not discuss this here. 
Now, the maximum likelihood estimate for @ is given by 


xy — #) = 0 


where y="), 022)", =) FRC Ee Sia)” andere Bene 
Under suitable regularity conditions, we have 


n*(6 — 0) > N([O, n (XTAX)~'], where A = D(I — D). 
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If we have X9 = X 6, + X, @, and consider testing the hypothesis 


we obtain the Wald statistic 


which is asymptotically equivalent to W,, under A). 


_ where 


& =U - X(XAX) XA 


The likelihood ratio test here is 


LR, = 25 |y, 


_ 5.2 Other Sampling Schemes 


a 

T 
a 
az 


log | 


T. 


t 


A aa 
p 2, #9 


) +a ~ 3) toe | 


e 


(i — #,) 


= 
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Suppose now that n~ 2X — rt) ~ N,V) and that Vis a consistent estimator of V. Here 
y is not necessarily a vector of 0’s and 1’s, but may in fact depend on the sampling weights 
and other adjustment factors. Estimating Vis usually possible since X’(y — z) is the sum of 
random observations and most sample designs admit a consistent estimator of the sum of (not 
necessarily independent) observations. To estimate V we use # instead of z in the estimate. 


Since asymptotically 


A 


we have that 


n*(6 — 0) > NO, n(X7AX) 'V(XTAX); 


see Binder (1983) for a detailed justification of this result. Now, a Wald statistic may be con- 
_ structed from the estimated covariance matrix for @,. 


Variable 


Table 3 
Logistic Regression Model for Explaining Use of Physician Services 


Age-Sex Interactions...... 
FBamilyalncome. 4... + sok. 
OCCIPAHON ganas acs eitia acs 
Occupation-Sex Interactions 
Niarital status © 402 de =: 
MedicaleEistony eee: .: 


Number of Disability Days 

| CommuUniiyg SIZ ert iy 
Provincial Physician - 

Population Ratios ...... 


Number of Health Problems.......... 


CEE OO AA) Meta), 


Type - Ane 
Categorical 4 19.232 
Categorical 1 12.494 
Categorical 4 36.001 
Categorical 5 14.642 
Categorical 3 8.614 
Categorical 3 11.501 
Categorical 3 45.752 
Categorical 2 36.700 
Quantitative 1 81.554 
Categorical 2 PAGIA Wis 
Quantitative 2 106.372 
Quantitative 2 29.052 
Categorical 2 11.75) 
Quantitative 1 0.540 
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j 


Example 4 


A logistic regression model was fit on 20,726 respondents from the Canada health surve: 
to explain use or non-use of physician services over a 12-month period. In total it was estimate 
that 77% of the population visited a physician at least once. The results are summarized i 
Table 3. For more complete details, see Binder (1983). The logistic model seemed to fit th 
data very well. | 


5.3 Qualitative Explanatory Variables 


The theory of this section was obtained by G. Roberts in an unpublished manuscript (Carletor 
University). Here the explanatory variables are all qualitative. We label the domains, {1, ..., J} 
We let p, be the survey estimate of the i-th domain proportion and N, is the estimate of thi 
size of the i-th domain, N,. Under the model, the expected proportion in the i-th domain i 1 
J, where 


log {f/(U-f) } = 470 


for a, known and @ an unknown parameter. We define A = [@,, ..., Gls and let Dy = 

dias {LNA WIN 
Under the model, the survey estimator of f = (f, ....f, )’ is given by his the solution tc 
A™D, (p—f) = 9. (5.1 


Since asymptotically 


where A = diag{N,f,d—f,). ~eesdV, i Li}. Wemhave 
n#(@'—*0.)>"N[0; G7 AA) 1A DY VY DIACAT ATA) 4 
whenever 7 - (De= 7 ) = N(O-Y,): 
Under independent binomial sampling, the covariance matrix reduces to (N/n)(A7AA)7' 


where 7 is the sample size. 
The likelihood ratio test for testing goodness of fit is 


LR, = (n/N) ¥,N, t plos(w/f,)+ -p) log{(—p)/A A}, 
where n is the sample size and N = L N.. Under H, this is asymptotically equivalent to 
= G/N) EN, — AY 140 = Al 
In general, the distribution of LR, will be that of L 6, Z?, where {Z,} are independent x‘! 


and {5} are the eigenvalues of N-'D,[A~' — A(A7A.A) 1A] Dy VpDy (A! — A(AA A)!" 


AD;'. By taking the expectation of W,,, and approximating 


LNG: | 
shies | 


where s = rank (A), we obtain 
| 
| 


E 4, = (n/N) Y Nv ff, (1-£)} 
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where v? = V{p,— f}. The {v”} may be computed using the relationship pfs 


UL = diag {4,0 -f,)} 444) '4'D, 1 — Pf). 7 


Example 5 


The data from the October 1980 Canadian Labour Force Survey was used to fit logistic (logit) 
models for the probability of being employed. The sample consisted of males aged 15-64 who 
were in the labour force and not full time students. A logit model, quadratic in age and in educa- 
tion, was fitted. Age-group levels were formed by dividing the interval [15, 64] into ten groups 
with the jth age-group being the interval [10 + 5/,14 + 5/],j = 1, 2, ..., 10. The midpoint 
| of each age-group was used as the value of the age for all persons in that age-group. Six levels 
of education were formed by assigning to each person a value based on the median years of 
_ schooling. Age by education classification led to the formation of 60 cells. 

Let z, = Pr{an individual in the ith cell is employed}, i=1, 2, ..., 60. We assume that 
0 < 7, < 1. Hence | — 7, represents the probability that the individual in the ith cell is 
unemployed. The model, considered for fit, was 

1. 


ogy = Bo + Bia, + Ba; + Bd, + Bi, (1) 
aa 


In 


Phe A. ce OU 


_ where a, and d, are the age and education variable values for the individuals in the ith cell. 
| _Using the survey estimates p, of z,, the values of Pearson’s statistic W,, and the likelihood 
| ratio statistic LR, were computed as W,, = 98.94 and LR, = 101.20. The upper 5% point of 
| the chi-square distribution, with 55 degrees of freedom, is 73.31. Using these values of W,, 
or LR, we would reject the model 1. These values of W,, or LR,, however, are appropriate only 
_if the sample was a random sample. 

The estimate average eigenvalue, L 6,/55, for testing goodness of fit for this data is 1.88. 
This would reduce W,, to 52.63 and LR, to 53.83. Hence, with this adjustment, we find that 

the data are Bovcinreut with the model (1). 

The use of the Wald statistic, (p — f)’([V"]” (@ — f ), for testing the goodness of fit was 
_also considered. Here we use the g-inverse of V since the matrix is singular. Some perturbation 
_| to the estimates of p,, when p, = 1, was necessary for computing the Wald statistic. It was 
_ found that the Wald statistic was unstable for our problem. Minor perturbations in the estimates 
_ of p led to considerable change in the value of the Wald statistic. Also the value of the Wald 
_ statistic is very large here due to instability in the estimated covariance matrix involved in its 
| calculation. The Wald statistic is at least 30 times larger than our adjusted Chi-squared values. 


6. SOFTWARE CONSIDERATIONS 


__ Advancement of computer technology has made data collection, storage and retrieval opera- 
_ \tions easy and efficient. Powerful generalized software systems, such as TPL, STATPAK and 
_ ESTIMATION SYSTEM, have been used to produce cell estimates and some of their variances 
fairly easily to users and analysts. As well a number of commercially available packages such 
‘as BMDP, SPSS and SAS are powerful analytic tools in certain contexts. However, the ability 
to perform analysis such as those described in this paper are limited. For example, in situations 
involving hypothesis testing or statistical inference, these packages assume that the data to be 
analyzed come from surveys with simple random samples. 

At present, an integrated software package, similar to the ones mentioned above, but designed 
for analyses of the type of data discussed in this paper, is not available. As a result, the researcher 
requiring a quick solution to his problem is usually forced to use existing statistical packages 
which may not be appropriate. 
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The alternatives are 


e use existing packages with modifications 
e use existing stand-alone software 

e write customized programs 

e use combinations of the above. 


For the analyses given in this paper, modifications to the MINI CARP program (Hidiroglou, 
Fuller and Hickman; 1980) were incorporated to obtain the results in Examples, 1, 2 and 3. 
For Example 4, a combination of PL/1 and SAS programs were developed. The analysis of 
the Labour Force Survey data (Example 5) used a combination of customized programs and SAS. 

For the above alternatives, some practical drawbacks have been experienced. they include: 


(a) If an existing package is to be modified, intimate knowledge of the package is often 

required; 

(b) Identical information may have to be duplicated on separate data files, as these alter- 

natives are not integrable like generalized systems; 

(c) Compared to an integrated ‘‘user-friendly’’ package, these alternatives lack elegance and 

operational efficiency as software; | 

(d) Comprehensive documentation is not generally available for specially written programs 

limiting the availability of software. 

Work is now ongoing to develop SAS based procedures for performing many of these analyses, 
Our ultimate goal is similar to that proposed by Shah (1981); namely, the development of an 
integrated software package for survey data analysis. This is a goal worth striving for, if we 
are to avoid the frustrations now being experienced by researchers who are faced with either 
developing their own software or using existing software which could lead to erroneous results, 
and conslusions. 


7. DISCUSSION 


We have examined a number of problems which arise when fitting models to categorical data 
which have been collected under complex sampling designs. The basic approach has been to 
derive the appropriate Wald statistic for the fitted model or to use the test statistic which is) 
motivated from multinomial-type sampling designs and find a suitable approximation to its 
null distribution. 

We have not addressed the issue as to whether one should really be taking a model- bad 
or design-based approach to begin with. Instead, we have concentrated on design-based 
inferences. | 

To put this issue into focus, let us reconsider the test of independence in a two-way con- 
tingency table. The question of independence arises if we are interested in whether knowing 
the value of variable Y, affects our knowledge about variable Y,. If it does not, for all the in- 
dividuals in the population, then we say the variables are independent. However, if we alsc 
know the value of Y,, it may turn out that Y, and Y, are no longer independent. This is par- 
ticularly important when Y, is a design variate (Gael as geographic stratum). Since desigr 
variables are usually cneway for all sampled individuals, we have one of two options: (a) wé 
can say that the question of independence is no longer relevant, or (b) we can marginalize oul 
Y,, and say that we are only interested in Y, and Y,, unconditionally. Assuming that we take 
A oproeeh (b), the results of this paper seem appropriate. In some cases it may be possible tc. 
test if Y, and Y, are conditionally independent given Y,. 

Theres isa furier difficulty, however. Suppose we are interested in the cell proportions 1, 
from a finite population of size N. If we were to take a census from this population, it is highly 
unlikely that we would obtain CF eg Y MLSE ; exactly. The best that we could hope for is thai 
some measure of association such as N Ey (ay a ath hae /n,,m,, 18 small. Note that evel; 


| 
f 
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under a super-population model of exact independence, we would not expect this measure of 
_association to be zero. Perhaps, we should instead be testing hypotheses such as 


H,: Measure of Association < C 


H,: Measure of Association > C. 


Further research is needed in this area. However, for practical circumstances where the sampl- 
jing fraction is not large, the methods given in this paper are suitable. 
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| PIERRE A. CHOLETTE! 


ABSTRACT 


i 
This paper presents a moving average which estimates the trend-cycle while eliminating seasonality from 


semi-annual series (observed twice yearly). The proposed average retains the power of all cycles which 
last three years or more; 90% of those of two years; and 55% of cycles of one year and a half. By 


‘comparison, the two by two moving average retains the power of respectively 75%, 50% and 25% of 


| 


the same cycles. 


‘KEY WORDS: Moving averages; Economic cycles; Spectral analysis. 


1. INTRODUCTION 


_ In some cases, semi-annual series exist for which there are no corresponding monthly data. 
_ {mn such instances, one cannot derive the seasonally adjusted semi-annual series from the monthly 
_ seasonally adjusted values. In addition, to our knowledge, there are no seasonal adjustment 
methods for semi-annual series. 
| This paper presents a moving average which eliminates seasonality and estimates the trend- 
rycle of semi-annual series. The approach of quadratic minimization used originates with 
Whittaker (1923) and was further developped by Leser (1961 and 1963), Cholette (1980), Schlicht 
1981) and others. 
The average derived has five terms and comprises a set of central weights for the semestres 
half-years) at the centre of series; and two sets of end weights, for the two first and last semestres. 
Consequently there is no loss of estimates at the ends of series, as with the two by four moving 
verage (used for quarterly series) for instance. 
. The spectral properties of the central weights prove to be superior to those of the two by 
wo moving average, which first comes to mind as a way of processing semi-annual series. The 
roperties of the end weights will also be examined. 


2. ILLUSTRATION OF THE AVERAGE 


Figure 1 shows the observed semi-annual original series z, (dashed line) along with the trend- 
ycle c, (solid line) estimated by the semi-annual cyclical average presented in this paper. As 
xpected, the trend-cycle behaves smoothly and displays short run cycles, namely a three-year 
ycle extending from the second semestre (half-year) of 1977 to the first of 1980. An estimate 
available for each observation, including the two first and last observations. The trend-cycle 
roduced by the two by two moving average (dotted curve) on the other hand does not yield 
ny estimate for the first and the last semestres. Furthermore, the two by two does not reach 
3 deeply into the cyclical peaks and troughs compared to the proposed average. 


Pierre A. Cholette, Time Series Research and Analysis, Statistics Canada, 25th floor, R.H. Coats Building, 
Tunney’s Pasture, Ottawa, Ontario, Canada KIA OT6. 
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Figure 1. 


Semi-annual seasonal series (----) and its trend cycle (——) estimated by the proposed semi- 


annual cyclical moving average and by the two by two moving average ("°° °) 


Table 1-A displays the exact values of the weights of the semi-annual cyclical moving average 
used. The first row gives the modified central weights pertaining to estimates 3 to 28 (in Figure 
1); the second, the end weights pertaining to the second-last estimate; and, the third, to the 


3. WEIGHTS OF THE AVERAGE 


last estimate. Table 1-B shows the central weights derived according to the methodology of | 


Section 6. We judged however these should be replaced by the modified central weights of Table 


1-A for reasons to be explained. 


Table 1-A 


Exact weights of the proposed semi-annual cyclical average 


Modified 
Central weights — 0.1000 0.2500 0.7000 0.2500 —0.1000 
Second-last 
Set of weights 0.0625  —0.2500 0.3750 0.7500 0.0625 
Last 
Set of weights — 0.0625 0.2500 —0.3750 0.2500 0.9375 
Table 1-B 


Unmodified central weights 


— 0.0625 0.2500 0.6250 0.2500 —0.0625 


| 
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4. SPECTRAL ANALYSIS OF THE AVERAGE 


The curves of Figures 2 and 3 represent the gain functions of the set of weights analysed. 
_ The gain value on the ordinate indicates the percentage of amplitude of the sinusoidal waves 
preserved, that is passed to the estimates, by the weights. The frequency of these waves is shown 
_on the abscissa and varies from 0 to 0.500. Frequency 0.500 corresponds to the annual wave 
of two semestres (1/.50), that is to stable seasonality. Moving seasonality is accounted for by 
a few neighbouring quasi annual frequencies: 0.467 and 0.483. 
Frequency 0.333 corresponds to a three (1/.33) semestre wave; frequency 0.250, four semestres; 
i 0.200, to five semestres; 0.167, six semestre, etc. Frequencies associated with waves of three 
semestres or more (left of 0.333 in figures 2 and 3) pertain to the trend-cycle of series and con- 
| stitute the target frequencies of the estimator. 
The frequencies between 0.333 and 0.467 exclusively are associated with fluctuations of 
periodicity less than one and a half years and superior to the quasi-annual seasonal frequen- 
) cies. They pertain to the irregular component of series. An ideal cyclical average should eliminate 
100% of these irregular frequencies, 100% of the seasonal and quasi-seasonal frequencies and 
preserve only the cyclical frequencies from 0 to 0.333 inclusively. 


a) Analysis of the central weights 


The solid curve of Figure 2 shows that the modified central weights of the semi-annual cyclical 
_ average preserves 100% of all waves of five semestres (2 years) and more: everywhere left of 
_ frequency 0.200 the curve is above 100%. By comparison, the two by two moving average, 
_ represented by the dotted curve, only preservs 65% of the 5-semestre waves and 93% of the 
_ 10-semestres waves. Furthermore, the modified central weights pass 55% of 3-semestre waves 
_ and 90% of 4-semestre (2-year) waves; against 25% and 50% respectively for the two by two. 
; Both sets of weights completely eliminate stable seasonality, with gain valued at zero for the 
_ seasonal frequency 0.500; and nearly all the moving seasonality. However, the two by two eliminates 
slightly more of the irregular frequencies than the modified central weights. When choosing bet- 
_ ween the two averages, one then faces the following trade-off: to let the estimates contain more 
_ cyclical movements but also more irregularity or less cyclical movements and less irregularity. When 
aseries is known to contain more cyclical movements (especially faster movements) than irregularity, 
the modifed central weights of the proposed average are certainly preferable to the two by two. 
The dashed curve of Figure 2 represents the gain of the unmodified central weights of the 
semi-annual cyclical average, as obtained from Section 6. At the cyclical frequencies, its per- 
formance proves superior to that of the two by two; but, inferior to that of the modified cen- 
ral weights. For instance, the latter reproduces 101% of the 5-semestre waves (frequency 0.200) 
Against 88% for the unmodified. The amplification of 5% (gain of 105%), at the 6-semestre 
‘frequency 0.167) wave with the modified central weights, seems to us preferable to a com- 
darable reduction of 6% (gain of 94%) with the unmodified set of weights. Indeed the analyst 
stands a better chance to detect an amplified signal in a series than a reduced signal. 


») Analysis of the end weights 
Ideally, the gains of the end weights should be identical to the gain of the central weights. 
‘Nsuch a case, end and central weights would have the same effect on the processed series (ex- 

 tept for possible phase-shifts). 

The gain of the weights for the second-last estimate (dotted line of Figure 3) is quite similar 
0 the gain of the modified central weights (solid curve). Note that the former is more similar 
Othe latter than to the set of unmodified central weights of Figure 2. This is the reason why 
ve modified the weights. 

| The weights for the second-last estimate preserve the cyclical frequencies and eliminate stable 
easonnality. However, they preserve 11 and 21% of the moving seasonality frequencies 0.467 
ind 0.483; and, even more of the noise frequencies. From the view point of the gain, the weights 
f the second-last estimate should yield less reliable estimates than the modified central weights. 
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FigaeZ 
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Figure 2. Gain functions of the modified —M—) and non-modified (----) central weights of the proposed | 
semi-annual cyclical average and of the two by two average (:°°°) 


The situation gets worse for the set of weights for the last estimate (broken line in Figure 3). 
Here, a strong amplification of some noise and fast cyclical frequencies is observed (gains | 
reaching up to 137%). Caution should then be exrcised in interpreting the estimate yielded by 
these weights. One should perhaps disregard the last estimate completely, for series which are 
reputed to be irregular (containing those magnified frequencies). 

As seen in Table 1, the end sets of weights are not symmetric. Consequently, they cause 
phase-shifts, which are compiled in number of semestres in Table 2 for certain selected fre- 
quencies. At the target cyclical frequencies, a small phase-shift is observed for the second-last 
weights. In this case, a cyclical wave of five semestres will be delayed by 0.09 semestres in 
the estimates; one of four semestres, by 0.16 semestres; and one of three semestres, by 0.28 | 
semestres; etc. 

The phase-shift reaches its maximum at the fundamental seasonal frequency (0.500). This 
does not matter however, since the frequency is totally eliminated by the weights. It does mat- 
ter a little for the moving seasonality frequencies 0.467 and 0.483, since they are not completely 
eliminated. | 


i 
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(++) and the last (----) estimates 


Table 2 


cyclical frequencies: 


0.100 (10 semestres) 
0.167 ( 6 semestres) 
0.200 ( 5 semestres) 
0.250 ( 4 semestres) 
0.333 ( 3 semestres) 


seasonal frequencies 
0.467 


0.483 
0.500 ( 2 semestres) 


Phase-shifts observed in number of semestres for the sets of 
end weights at certain selected frequencies 


second-last 


0.01 
0.05 
0.09 
0.16 
0.28 


0.46 
0.48 
0.50 


last 


0.01 
0.05 
0.05 
0.00 
OLE7 


0.45 
0.47 
0.50 
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‘gure 3. Gain functions of the modified central weights (——) and of the weights for the second-last 
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5. GRAPHICAL ANALYSIS OF END ESTIMATES 


Figure 4 displays the preliminary estimates derived using the two sets of end weights for years 
1968 to 1980, accompanied by the corresponding central final available estimates. Figure 4 a) 
shows the end estimates falling in the second semestre; and 4 a), in the first semestre. (One single 
plot would have been too crowded.) 

If the central estimates are considered as true (or at least more reliable, the end estimates 
are seen to cause five false signals: in 1968 (arrow in fig. 4 b), in 1972 (4 a), in 1974 (4 b), 
in 1975 (4 a) and in 1976 (4 b). A false signal is said to occur here when the end estimates 
show a change in the direction of the trend-cycle and when that change is later contradicted 
by the central final estimates (becoming available with new observations). These false signals 
tend to appear when the series slows down in one direction and resumes the movement in 
the same direction. When there is a strong change of direction like in 1978, this does not 
seem to occur. | 


QOCF (CCS CON OM Gal) 2730 7/4 7,5 en OMen a(t Omen Om SO 


Figure 4. Semi-annual seasonal series (----); preliminary estimates of its trend-cycle by the ends weight: 
G ) of the proposed semi-annual cyclical moving average a) for the second semestres and b) for the 
first semestres; final estimates (----) by the central weights of the average. 


| 
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__ If the estimates derived by the last set of weights were omitted, many false signals would dis- 
appear. However, the estimates sould become less timely. This illustrates the statistician’s dilem- 
'ma between the timelyness and the reliability of estimates under any estimation method. (In 
practise, a serious analyst would wait for at least one confirmation of a signal before believing it). 
Apart from the five false signals mentionned, the preliminary estimates display a movement 
which is very similar and sometimes undistinguishable from that of the final estimates. 


5. CALCULATION OF THE WEIGHTS OF THE 
i SEMI-ANNUAL CYCLICAL AVERAGE 


__ The observed series z, comprises the trend-cycle c, to be estimated and a seasonal-irregular 
mesidual s, + e, (= Z, — ¢,): 


Rie Cit sade) foe LL Ri sehiSe (1) 
Following the approach of Leser (1961 and 1963) and of Cholette (1980), the desired trend- 
>ycle minimizes the quadratic sum of fourth differences (first term of (2)). On the five-semestre 
estimation interval, the component as much as possible approximates a time polynomial of the 
chird degree. This specification allows for a full economic cycle with its four phases of expan- 
sion, turning-point, recession and recovery over the interval. 
The seasonal-irregular residual (z, — ¢,) minimizes the quadratic sum of first seasonal dif- 
“erences taken on corresponding semestres (second term of (2)). This specification means that 
he seasonal-irregular residual of one semestre should resemble that of the same semestre in 
he neighbouring year as much as possible. 
| Furthermore, the seasonal-irregular residuals minimize the quadratic sum of their sums on 
_ wo consecutive semestres (third term of (2)). This criterion indicates that the seasonality of 
_ wo neighbouring semestres should cancel out and that the irregularity should not affect the 
evel of the desired trend-cycle. 
| The three criteria specified for the components combine into the following objective function: 
| 


5 
iQ = L [Ca AC egeich: \OCn ga trighy neck Ra) 


i | (2) 
| Ge) ey) teen Gat Gtlty eel 
' quation (3) can be rewritten in linear algebra: 
| Vis) St AIA Cre (Ze OC) BBE] Olt Zs OW FZ} C) ie 


Gee) bet 2G (Goer Z) 


here A, B and C respectively stand for the matrix operators of quadruple differences, first 


tsiiaol Dal Reinet 
Pete! 406 4 |B = a oe Maa call a lai Brie aye CF 
ia eroleae cai 0; &, fused 
‘The normal equations associated with (3) read 
aF/dC = 2HC + 2G(C — Z) = 0 (4) 


d imply solution: 
C = (H + G)"'GZ = WZ (5) 


164 Cholette: Estimating Economic Cycles 


The third central row of matrix W contain the non-modified central weights of the semi- 
annual cyclical average ot Table 1-B; and the fourth and fifth row, the end weights of rows 
2 and 3 of Table 1-A. 


7. HISTORY OF MOVING AVERAGES BY QUADRATIC MINIMIZATION 


This approach of quadratic minimization originates with Whittaker (1923). Leser (1961, 63) 
showed how quadratic minimization could be applied to develop cyclical moving averages. 
Cholette (1980) proposed substitutes for the two by twelve and the two by four moving averages. 
These substitutes were incorporated into the Dagum (1980) seasonal adjustment programme 
as optional. 

The semi-annual cyclical average presented in this paper could also be incorporated in a 
seasonal adjustment method of the X-11 type. This would allow seasonally adjusting semi-annual 
series and the calculation of the seasonal factors by means of the seasonal moving averages. 
usually applied for monthly and quarterly series. 


8. CONCLUSION 


This paper presented a 5S-term moving average which eliminates seasonality from semi-annual 
time series. The estimator reproduces the economic cycles more exactly than the two by two 
moving average. The two by two also has the disadvantage of not providing any estimate for 
the first and last semestres of the series. 
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ABSTRACT 


This paper discusses the use of matching between files of comparable data in the evaluation of non-sampling 
error. As an example of the technique, the data quality evaluation of the 1981 Canadian Census of 
» Agriculture is described and some results presented. 
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1. INTRODUCTION 

As the use of probability sampling in data collection has evolved, the evaluation and control 
of sampling errors has been a constant concern. Extensive research has been devoted to the 
‘design of sampling schemes which would reduce sampling error and facilitate its measurement. 
In many situations, however, major portions of the survey error arise not from sampling, but 
from the effects of other components of the data collection operation. In censuses particularly, 
| | in which data are obtained through 100 percent enumeration of the population of interest, sampl- 
| ing error is nonexistent. Instead survey error is due entirely to the influences of respondents, 
_ interviewers, coders, keyers, and others during the collection, capture, and processing stages 
if of the survey operation. As the impact of these non-sampling errors on data quality has become 
if ‘more fully understood, the development of techniques to control and measure them has gained 
1 ‘in importance. 


. 2. MODELS FOR SURVEY ERROR 


Early papers on total survey error, such as that by Deming (1944), outlined the potential 
_ sources of error and discussed the need to consider their varying effects when planning data 
collection operations. As the study of survey error developed, general models were propose 
‘by Hansen et al. (1951), Sukhatme and Seth (1952), Hansen, Hurwitz, and Bershad (1961), 
; and others to describe the components of sampling and non-sampling error. Studies were con- 
ducted on the correlations between errors which result from influences such as interviewers or 
‘coders, and methods were developed for measuring their effects. Fellegi (1964) presented a detail- 
_ 2d model which included correlations between numerous error sources. 

Other models have followed which consider both single and correlated non-sampling errors 
and propose methods for evaluating them. Some examples include the U.S. Bureau of the Cen- 
sus survey error model described by Nisselson and Bailar (1976), the discussion of measure- 
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ment errors by Cochran (1977), and the model of survey error presented by Andersen et al. 
(1979) which was based on an earlier model by Kish (1965). In a recent paper Hartley (1981) 
described a model with terms for interviewer, coder, and respondent errors, and proposed a 
sample design to facilitate estimation of these errors. 

Throughout the literature the components of non-sampling error have been categorized in 
a variety of ways. In this paper we will divide non-sampling error into two elements, coverage - 
error and response error. A coverage error will be defined to have occurred when a unit which 
satisfies the definition of the universe of interest is missed or counted more than once, or a 
unit not belonging to the desired universe is included. Coverage errors cause the invalid inclu- 
sion or exclusion of all data for the incorrectly enumerated unit. As a result they may influence 
the estimates for any or all data items. 

Response errors will be defined as affecting the values of individual items within the data 
for units which have been correctly included in the enumeration. They may have arisen at the | 
initial collection of the data or during subsequent processing stages; potential sources include | 
misinterpretation of a question by the respondent, total or partial non-response, the influence 
of the interviewer, and data capture or coding error. 


3. EVALUATING NON-SAMPLING ERROR 


In most survey error models, error is defined as the difference between the true value and 
the collected value for the particular data item. Thus in order to evaluate non-sampling error, 
one would in theory propose to compare the data collected by the survey with the true values 
for the item of interest. In practice, however, the true values are seldom known, if in fact they 
even exist. Instead the survey data must be compared to estimates from an alternate source 
which is believed to provide the closest available approximations to the true values. 

In determining the data source most suitable to represent the unknown true values, a number 
of factors must be considered. The alternate data must be collected independently from those 
of the survey of interest. Optimally, the definitions and concepts employed in the collection 
of the data, and the reference periods to which the data applied, would be equivalent for the 
two sources. The universe covered by the alternate data would be the same as that of the survey, 
or comparable subuniverses would be identifiable. The purpose and methods of collection of 
the alternate data, and of any subsequent processing or updating stages, would be fully 
understood. Perhaps most important of all, the data would be of high enough quality to act 
as a standard against which the survey data could be compared. 

In practice, of course, all of these conditions are seldom satisfied by a single alternate data 
source. In some cases one source may be best for a particular subpopulation while another is 
preferable for the remainder of the universe. Adjustments may be possible to remove the ef- 
fects of differences in reference dates or definitions of variables between the two data sources. 
For the majority of cases, however, even the best estimator of the true values will involve ma- 
jor failures of some of these conditions, and their influence on the comparison may not be 
measurable or even identifiable. 

Approximations to the true values may be obtained from a variety of sources. Estimates 
from one census or survey may be compared to those from another. Examples include the com- 
parisons of the Current Population Survey and the U.S. Census of Population, the Labour 
Force Survey and the Canadian Census of Population, and the Agriculture Enumerative Survey 
and the Canadian Census of Agriculture (Statistics Canada 1979). Demographic projections 
have also been employed to approximate the true values, such as in the evaluations of the Labour 
Force Survey and the Canadian Census of Population described by Fellegi (1973). 

Administrative data are also being used more and more in evaluation studies as respondent — 
burden beccomes an increasing concern. Estimates from income tax, family allowance, motor 
vehicle licence, agricultural marketing board, and other files may provide approximations to 
the required true values. Income tax files, for example, are currently being studied at Statistics 
Canada for use in the collection and evaluation of farm income data. 
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As these few examples imply, a wide range of alternate data sources have been used as stan- 
dards of comparison for survey estimates. However, while macro-level comparisons provide in- 
‘dications of the total error in the survey results, they cannot identify or measure the components 

of the error. The effects of coverage and response errors cannot be identified. Hence additional 
methods are required which facilitate more detailed investigations into the total error observed. 

One technique, which can provide the analyst with a wealth of information on errors and 
their sources, links the survey results at the individual record level to a file of comparable data. 
Using the matched records, one-to-one comparisons can be made between the values reported 
for the survey and those from the alternate data source, which are assumed to represent the 
true values. Cross-classifications over other data items provide insight into the characteristics 
of units displaying certain types of inconsistencies. As well, the study of records which could 
not be matched can indicate areas of potential over or undercoverage by the survey. 

In this paper we will consider the use of matching in the evaluation of survey non-sampling 
error. The strengths and weaknesses of the technique, and the types of studies which it makes 
possible, will be discussed. As an example of the use of matching, the data quality evaluation 
of the 1981 Canadian Census of Agriculture will be outlined and some results presented. 


4. THE USE OF MATCHING 


Study of the literature on the evaluation of total survey error reveals many studies which 

have made use of matching. As with the macro-level comparisons, a wide variety of comparable 
data sources have been employed. Post enumeration surveys, aimed at collecting data of a higher 
quality and in more detail than the original survey, have been conducted in a number of situa- 
tions. Examples include reinterview by appraisers in a study on the reporting of the market 
value of homes (Kish and Lansing 1954), the post enumeration survey used to evaluate the U.S. 
Census of Agriculture (U.S. Bureau of the Census 1982), the Labour Force Survey reinterview 
orogram (Tremblay, Singh, and Clavel 1976), and the Vacancy Check Operation of the Cana- 
dian Census of Population and Housing (Statistics Canada 1980). 
_ Independently-collected censuses and surveys have also been linked in order to evaluate data 
aaity. Matches have been performed, for example, between the Labour Force Survey and 
_ he Canadian Census of Population (Krotki 1980), the Current Population Survey and the U.S. 
_ Densus of Population (U.S. Bureau of the Census 1964), and the Agriculture Enumerative Survey 
ind the Canadian Census of Agriculture (Statistics Canada 1979). 
_ With the concern for reducing respondent burden, administrative data have been used in- 
_ ‘reasingly in evaluation studies. Records of doctors and hospitals have been matched to health 
_ urvey results by Andersen et al. (1979) and Horvitz (1981). Immigration and birth records have 
_ »een employed in the Reverse Record Check Content Study of the Canadian Census of Popula- 
ion (Krotki 1980), and tax records of the IRS have been used to study response errors in the 
_J.S. Census of Population (U.S. Bureau of the Census 1970). Other examples of linkage with 
 dministrative data include the evaluation of agricultural survey results by Faulkenberry and 
_ Yortora (1981) and the reporting of sensitive topics by Marquis, Marquis, and Polich (1981). 
| In general, the quality of an evaluation study based on a record linkage operation depends 
‘n two major factors: 


| ») the quality of the data on the alternate source to which the survey is matched, and 
) the uniqueness of the identifiers used for the match, and the accuracy of the match 
_ technique itself. 


_, Firstly, by definition of the study objectives, the alternate data are to act as approximations 
_) the true values. Random errors in the data which tend to cancel one another out over all 
cords do not noticeably affect macro-level comparisons. However such errors can have a serious 
ffect on studies conducted at the individual record level. 

_ Some data bases chosen to act as standards for comparison may be assumed to be free from 
tror. Birth records, for example, provide very accurate data on place of birth and age. Other 
fata sets may be known to contain certain response or coverage errors, but if the errors are 
veasurable they can be taken into account in the analysis, and the assumption of no error re- 
lains valid. In many cases, however, the comparable data are subject to errors which cannot 
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be completely identified or measured. In these situations, the best approximation to the true 
values is provided by the data set which is least affected by error. Data which have been col- 
lected using more accurate methods or better trained staff than the survey of interest may be 
assumed to contain less error, and hence can provide a reasonable basis for comparison. — 

The second major factor affecting the quality of the study is a function of the match opera- 
tion itself. In some situations, each member of the population will have been assigned a unique 
identification number which has been accurately stored on both files. Linking the records would 
then be a straightforward process of matching on these unique identifiers. At the other end 
of the scale, the best available identifiers may be non-unique characteristics, such as name, which 
are prone to the introduction of error during data collection or capture. | 

The linkage algorithm itself can also have an impact on the quality of the match, particular- 
ly when the keys or identifiers are less than perfect. The algorithm may tend to allow invalid 
matches between records with similar keys, or may prevent valid matches when the keys differ 
due to minor errors or omissions. The extent to which such errors occur can have a significant 
effect on the composition of the files of matched and unmatched records. 

Other factors which affect the comparability of the two sets of data, as for macro-level studies, , 
include differences in collection date and method, concepts and definitions, and reference period. 
Due to the greater detail of investigation and cross-classification over related variables which 
is involved in micro-levels studies, such differences can have a much greater impact on the analysis) 
than for the macro-level comparisons. 

In order to study the use of matching in the analysis of non-sampling error, we will now 
consider the example of the data quality evaluation of the 1981 Canadian Census of Agriculture. 
For this study, independently-collected agricultural data for macro and micro-level comparisons 
were provided by the Agriculture Enumerative Survey (AES) and the Farm Enumerative Suri | 
(FES), annual probability surveys conducted by Statistics Canada. 


5. COMPARING THE CENSUS AND SURVEYS 


The Canadian Census of Agriculture was conducted on June 3, 1981, sharing field opera- 
tions with the quinquennial Census of Population and Housing. Data were to be collected for 
every census farm in Canada, defined as any farm, ranch or other agricultural operation which 
received $250 or more from the sale of agricultural products during the twelve months prior 
to census day, or which had the potential to produce that value in the next twelve months. Dur- 
ing drop-off of the population and housing questionnaire, the census representative was to ask 
at each household whether any member operated a farm or other holding which satisfied the 
above definition. If so, a Census of Agriculture form was left to be completed by the operator. 

In order to improve coverage, results of the 1976 Census and subsequent agricultural surveys 
were used to identify farms which were major producers of one or more specified agricultural 
commodities. The census representatives then had to account for each of these ‘‘specified farms” 
located in the area to be enumerated. 

The questionnaire, delivered prior to June 3 to the operator of each census farm, was to 
be completed by self-enumeration on census day. Items covered in the census included crops, 
livestock, land use, sales, expenses, and other areas of interest to the public and private sectors. 
(Further details on the methodology and content of the 1981 Census of Agriculture may be 
obtained from the publication Statistics Canada (1982).) 

The Agriculture Enumerative Survey (AES) and Farm Enumerative Survey (FES) together 
covered the majority of Canada’s agricultural land. The FES enumerated the Prairie provinces 
of Manitoba, Saskatchewan, and Alberta plus the Peace River district of British Columbia, 
and the AES covered the remainder of British Columbia and the provinces of Prince Edward. 
Island, Nova Scotia, New Brunswick, Quebec, and Ontario. The survey universe consisted of 
agricultural holdings which satisfied the census farm definition described above. However it 
excluded types of organization which were of marginal economic influence, such as institutional 
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. farms, and areas which contain little or no agricultural activity, such as urban cores. In order 
_ to provide comparable universes for the evaluation, the census file was adjusted by removing 
operations of these types. The deletions consisted of only 2.8 percent of the farms and 1.8 per- 

_ cent of the total farm area from the complete census file. 

The probability surveys collected data on the same major agricultural variables as the cen- 
} sus, such as crops, livestock, land use, and operating expenses, and used similar concepts and 
_ definitions. Some differences existed in wording and format, and in the instructions on what 
| to include or exclude, for particular questions. As will be indicated in the discussion of the 
_ results, the effect of these inconsistencies had to be taken into consideration when comparing 
| data from the two sources. 

The AES and FES were conducted on July 1, 1981, approximately one month after the June 
| 3 census date. Some data, such as farm expenses for the previous year, were expected to be 
_ relatively unaffected by the difference in reference data. However, other items were more likely 
_to change between June 3 and July 1. The effect was expected to be particularly significant 
_ for livestock items, due to the constant fluctuations in inventories caused by birth, deaths, pur- 
chases, transfers, etc. As a result, operators responding to the survey were asked to indicate 
_ the changes in numbers of cattle and pigs between June 3 and July 1. Evaluation indicated that, 
while the data obtained were of some use in reconciling the differences due to reference date, 
_ they were subject to high non-response and questionable accuracy. Hence, the comparison of 
these and any other variables which would tend to be influenced by date of response had to 
take into consideration the difference in reference dates between the census and survey. 

/ The samples for the AES and FES were selected from an area frame of agricultural enumera- 

‘tion areas, supplemented by a list frame of farms which were major producers of certain im- 

portant commodities. Data collection was performed by trained enumerators during a personal 
| ‘interview with the operator of each selected farm. Following the necessary processing stages, 
an estimation procedure was applied to scale the counts up to the level of estimates for the 
entire population of interest. (Further details on the sample design are found in Statistics Canada 
_ (1984) and Phillips (1978).) 
| The survey estimates were subject to the same types of non-sampling errors as those from 
the census. However, due to the concentration on a smaller number of holdings, and the im- 
i proved control of operations which was thus possible, it was expected that these types of errors 
_ would have a lesser impact on the surveys. Hence the surveys provided acceptable approxima- 
tions to the true data values. On the other hand, the survey estimates were affected by samp-- 
- ling error, which had to be taken into account when making comparisons with macro-level 
estimates obtained from the census. 


_ 5.1 Macro-level Comparisons 


_ Prior to the evaluation using the matched file, estimates from the complete census and survey 
data files were studied. Since the two vehicles covered comparable universes, these macro-level 
comparisons for provinces and regions provided initial indications of census coverage. By com- 
varing census point estimates with survey 95 percent confidence intervals for totals of livestock, 
srop acreages, and other items, areas of potential over or underestimation were identified. Fur- 
her investigation of the macro-level differences was then initiated to determine if they were 
vonfined to particular categories of the items of interest. The macro-level studies, in addition, 
_ orovided the experience and familiarity with the two sets of data which were required for the 
letailed analysis which followed. 

| Asan example of the results of the macro-level comparisons, Table | presents estimates for 
Sanada for the number of farms, total farm area, and land use. A significant difference be- 
‘ween census and survey estimates was observed for total farm area in Canada, yet the size 
ind direction of differences varried greatly among the component land use categories. Census 
‘stimates for classes of improved land differed from the survey estimate by as much as 25.5 
it 7.7 percent for other improved land to as little as -3.4 + 2.2 percent for cropland. The 
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Table 1 


Comparison of Census and AES-FES Estimates for Number of Farms, Area 
and Land Use (in thousands of acres), 1981, Canada* 


Item Census Survey Percent 
Estimate° Estimate® Difference? 

Total number of farms 309,410** 319,476 =°3.2°+'2:6 
Total area of farms 159,866" 175,543 — 8.9 + 2.4 
Improved land 112,390 114,610 SO) a> Ws) 
Cropland WSS 78,211 eae B\ lucie )-9) 
Improved pasture 10,523" 9,460 10 ye ae ee) 
Summerfallow 23,827" 24,939 — 4.5 + 3.7 
Other improved land 2,509" 1,999 Q5e5 7 
Unimproved land AeA 60,933 DD te 43 
Woodland Seo lite Weis — 53.7 + 3.9 
Other unimproved land 39,265" 43,182 el i yee) 


@ Excluding Newfoundland, Yukon and Northwest Territories. 

> Excluding specified marginal areas and farms not belonging to the survey universe. 

© Census and survey totals may not equal the sum of the components due to rounding. Survey 
estimates for Canada are based on a sample of 18,327 farms. 


; Census Estimate —- Survey Estimate f 
4 Percent Difference = ( - y ) x 100; the percent dif- 
Survey Estimate 


ference may not be consistant with the totals represented due to rounding. The indicated con- 
fidence interval, resulting from the sampling error in the;survey, is equal to 
census estimate 


+ 2 x (survey coefficient of variation) x : 
survey estimate 


© An asterisk, identifying a significant difference between estimates, is indicated when the census 
estimate lies outside the survey 95 percent confidence interval. 


major discrepancies in land area, however, were concentrated in the categories of unimproved 
land, particularly woodland. Further analysis into the reporting of woodland, which was prom- 
pted by these results, is discussed in section 5.5. 

Macro-level comparisons also included the study of estimated frequency distributions prepared. 
from the census and survey files. Distributions of the estimated number of farms over variables 
such as type of organization, land area, area of cropland, and sales were compared. Differences 
in the distributions identified possible over or undercounting of farms with particular 
characteristics. 

Table 2 presents the census and survey frequency distributions by type of organization 
for the estimated number of farms in Canada. No significant differences were observed 
between the estimates for individual or family farms or corporations. However further study 
was initiated into the coverage of partnerships on the basis of the discrepancies noted for this 
category. 

The limitation of the macro-level comparisons for evaluation of coverage was the inability 
to separate the effects of response errors from the effects of coverage errors. For example, the, 
differences between census and survey estimates for improved land categories, shown in Table 1, 
seemed to exhibit too much variation in direction and magnitude to be the result of coverage 
errors alone. The discrepancies for woodland might also have been caused by factors other than 
coverage. Perhaps differences in field procedures or questionnaire format had resulted in in- 
consistencies between the census and surveys in the inclusion or exclusion of land of questionable 
agricultural value, or the classification of certain categories of land use. The micro-level match 
provided the needed mechanism for investigating these types of issues. | 
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Table 2 


Comparison of Census and AES-FES Estimates for Number of Farms by 
Type of Organization, 1981, Canada® 


Type of Organization Census Survey Percent 
Estimate Estimate® Difference 
Total number of farms 309,410* “ 319,476 — 3.2% 2:6 
Individual or family 
farm 268,199 267,396 03 «By 8.0 
Partnership 
— with a written 
agreement 11,160° 15,908 —29.8 + 16.7 
— with no written 
agreement 17,646" 22,855 —22.8 + 10.8 
Corporation 11,744 12,160 4 a Org 
Other type of 
organization 661" 1,142 — 42.1 + 13.0 


For footnotes, see Table 1. 


5.2 The Micro-level Match 


Past experience with other agricultural censuses and surveys has indicated that even the most 
vareful attention to quality cannot entirely prevent response errors. Despite all attempts to pro- 
fide clear, unambiguous questions, problems such as differing interpretations of certain 
jgricultural terms across regions of Canada, or a lack of consensus on the appropriate classifica- 
‘ion for certain types of land use, influence the data collected. Misinterpretation is particularly 
| sommon for items which are of marginal economic or agricultural value,or which do not apply 
‘0 most respondents. The micro or record level match with the AES-FES files provided the 

| ‘means to evaluate the impact of response errors on the 1981 Census of Agriculture. 

| The match between the Census of Agriculture and the AES-FES was based on the operator 
_ tame, address, telephone number, and postal code for each holding. The link was performed 
_ nthirteen stages, each requiring a match on a different combination of the identifiers or their 
_ omponents. At each stage of the procedure survey records which has not yet been matched 
A vere identified, and the census file was searched by computer to locate the corresponding records. 
4 or each survey holding, the specified matching variables or keys were compared character by 
haracter with those of the census records which had not yet been linked. A match was iden- 
fied if all characters of the matching variables were equal. At the Canada level, a computer 
latch rate of 75.7 percent was achieved for the 18,327 survey records. 

It was inevitable that, for a certain number of survey records, no census farm would be iden- 
fied by the computer. Discrepancies in spelling of names and addresses, which had arisen dur- 
z ag collection or capture of the census or survey data, prevented links in many cases. For exam- 
le, J. Smith might have been reported on the census as opposed to J. Smyth on the survey, 
ames Smith as opposed to Jim Smith, or St. Catherines rather than St. Catharines. Partner- 
| hips or corporations for which one operator had responded on the census but a different part- 
» =r or manager had been interviewed by the survey could not be matched by a computer link 
1 operators. Similarly, records for holdings which had changed operators between the census 
hd survey collection dates could not be linked by computer. 

‘In order to improve the match rate, and eliminate the possible biases in the matched file 
ich might have resulted from those operations which could not be linked by computer, a 
nual resolution process was initiated. Using additional data from the questionnaires, such 
corporate or farm name, addresses and names of partners, land description of the holding, 
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and comments, clerical staff attempted to identify the corresponding census farm for each un- 
matched survey record. Of the 4,459 unmatched survey farms which remained following the 
computer link, 3,228 were matched during the manual resolution process. At its completion, 
93.3 percent of the total 18,327 AES and FES records for Canada had been linked to census 
operations. 

With further input of time and resources, it may have been possible to link some of the re- 
maining 6.7 percent of the survey records to the census data base. However, in many cases 
the needed identifiers has not been collected on either the census or survey, and would have 
required investigation of administrative records or contact with the operators themselves. It 
was felt that the possible benefits were not sufficient to warrant the expenditures required, and 
no further manual resolution was attempted. 

The studies which were facilitated by the record linkage can be grouped into two main types, 
those based on the unmatched survey records and those using the matched census-survey pair. 


5.3 Studies of the Unmatched Records 


In order to study the characteristics of census undercoverage, the unmatched records were 
assumed to be representative of the farms which should have been enumerated by the census 
but were missed. It was known that the unmatched records overestimated the number of missed 
farms, due to certain conditions of the data sources and the matching algorithm. For example, 
it was probable that some records on the survey file had been covered in the census, but could 
not be matched by either computer or manual means due to missing or invalid name or address 
data. 

Because of the resulting potential for overestimation of the land and commodities missed 
by the census, one had to proceed with caution in using the estimates produced from the un- 
matched records.Nonetheless the Canada level estimates were most valuable as initial indicators 
of the characteristics of the farms which were underenumerated by the census. 

In the first stage of the study, sample expansion factors were applied to the unmatched records 
to produce commodity estimates for the ‘‘missed farms’’. These were then compared to the 
commodity estimates for the entire survey universe, and the fraction of the total estimate whick 
was accounted for by the ‘‘missed farms’’ was calculated. This fraction was then compared 
with the fraction which the missed farms comprised of the total estimated number of farms. 
For example, Table 3 shows that the unmatched file contained only 4.4 percent of the estimated 
total farm area and 3.9 percent of the cropland, whereas it was responsible for almost 9.7 per- 
cent of the total estimate of farms. These results provided an initial indication that the ‘‘missed¢ 
farms’’ were not representative of the complete universe, but were smaller than average in terms 
of land area and other characteristics. This implied that the extent of undercoverage could noi 
be measured by the number of farms missed alone. Instead, the characteristics of the missec 
farms over particular commodities had to be considered. 

To provide further insight into the characteristics of undercoverage, frequency distribution: 
of the estimated number of missed farms were prepared over classes of land area, sales, livestock. 
and other commodities. Comparison with similar frequency distributions for the entire surve) 
universe indicated that the missed farms had a higher proportion of holdings with small acreage: 
and low sales. It can be seen from Table 4, for example, that 42.3 percent of the estimatec 
missed farms reported less than 70 acres of total farm area, as compared with 15.8 perceni 
of the complete survey population. Sales of less than $1,200 were reported by an estimated 27.1 
percent of the missed farms, but only 7.3 percent of the survey universe. 

The frequency distributions were also compared by considering the ratio of the unmatchec 
estimate to the estimate for the entire survey universe, that is, the fraction of the total survey 
estimate accounted for by the unmatched farms. As shown in Table 4, the unmatched file con: 
tained 36.5 percent of the estimated farms with less than 10 acres of land, the smallest sizé 
range presented as comapred with only 4.3 percent of those in the largest range of 760 acre‘ 
or more. Similarly 36.8 percent of the estimated farms with sales less than $1,200 were obtainec | 
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Table 3 


Comparison of AES-FES Estimates for Total Farms and Unmatched Farms, 
Area and Land Use (in thousands of acres), 1981, Canada@ 


Item Estimate from Estimate from Percent of the 
Total Unmatched Total Estimate 

AES-FES AES-FES Accounted for 

File? Records° by the Un- 

matched Farms 

Number of Farms 319,476 30,975 9.7 
Total Farm Area 175,543 7,768 4.4 
Total Improved Land 114,610 4,502 329 
Cropland 78,211 2,792 3.6 
Improved Pasture 9,460 603 6.4 
Summerfallow 24,939 1,004 4.0 
Other Improved Land 1,999 104 S22 
Total Unimproved Land 60,933 3,266 5r4 
Woodland 17,751 1325 ahs: 
Other Unimproved Land 43,182 1,941 4.5 


* Excluding Newfoundland, Yukon and Northwest Territories. 
> Survey estimates are based on a sample of 18,327 farms. 
© The unmatched file contained 1,231 farms. 


Table 4 


Percentage Distribution of AES-FES Estimates for Total Farms 
and Unmatched Farms by Total Farm Area and Total Value of Agricultural 
Products Sold During 1980, 1981, Canada* 


Item AES-FES AES-FES Percent of the 


Estimate of Estimate of Total Estimate 

| | Total Number Number of Accounted for by 

of Farms? Unmatched the Unmatched 

| Farms* Farms 

i | Cumulative Cumulative Percent 
Percent Percent 


Total Farm Area 


Under 10 acres 35 1392 36.5 
I 10 - 69 acres 15.8 42.3 22.9 
70 - 399 acres 62.8 85.0 8.8 
400 - 759 acres 78.4 90.5 3.4 
| 760 acres and over 100.0 100.0 4.3 


Total Value of Agricultural 
Products Sold 


Under $1,199 ais} Did 36.8 
$ 1,200 - $ 2,499 1235 41.2 26.5 
2,500 - 9,999 29.6 65.4 ie es 
10,000 - 49,999 67.8 88.3 5.8 
50,000 and over 100.0 100.0 Ze) 


@ Excluding Newfoundland, Yukon and Northwest Territories. 
Survey estimates are based on a sample of 18,327 farms. 
©“ The unmatched file contained 1,231 farms. 
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from the unmatched file, compared to 3.5 percent of those with sales of $50,000 or more. 
In summary, the results of the study of unmatched records were able to provide concrete 

evidence that the holdings missed by the census tended to be smaller than average in terms of 

agricultural production and value, a theory which had been widely held but not proven. 


5.4 Studies Using the Matched File 


The matched file was composed of agricultural holdings for which census and survey records 
could be linked by the computer and manual processes described. Since these census and survey 
values were assumed to have been collected from the same set of holdings, the effects of coverage 
differences were removed. In addition, the influence of imputation was lessened by excluding 
records for which census or survey data had been entirely imputed due to non-response. Hence | 
the nature and extent of potential response differences between the two data collection vehicles | 
could be studied. Prior to discussing the results of the matched record studies, however, a number 
of limitations which existed within the matched file, and which influenced the evaluation pro- 
cess, should be described. 

Although every effort was taken to lessen the chance of spurious matches, it is possible that 
a small number of survey farms may have been linked to the wrong census holding due to. 
similarities in name or address. In the case of extremely large agricultural operations which 
made a significant contribution to provincial commodity totals or land areas, linkage to the 
wrong census operation could noticeably skew the results. A detailed study of a sample of mat- 
ched records, undertaken to determine the quality of the computer link, had not been com- 
pleted at the time of the evaluation. As a result, the potential influence of spurious matches. 
had to be considered when studying results from the matched file. . 

The second limitation on the matched file analysis affected the comparison of total counts 
from the census and survey data. The matched records consisted of a subset of the non-self-| 
weighting sample of the AES and FES, since they contained only the survey farms which could 
be linked to census records.It would have been preferable to apply sample expansion factors’ 
to produce weighted estimates for the matched file. However, the expansion factors for the 
area frame were calculated using reported land use values from the survey, and it was as yet 
undetermined whether the factors were valid when applied to census data from the matched 
file. Census-related expansion factors could not be calculated using the census data, since one 
component of the factor, the farm area inside the selected segment of land, was collected only 
by the survey. As a result of this uncertainty regarding the application of survey expansion fac- 
tors to census matched data, it was decided to restrict the analysis to unweighted census and 
survey totals from the matched file. (Study of the use of weighted estimates from the matched 
file was underway at the time of writing, but no conclusions had yet been reached.) 

Despite the limitations of a non-self-weighting sample and the possible existence of spurious 
matches, the matched file proved to be a valuable evaluation tool. When matched totals iden- 
tified discrepancies between census and survey values, further detailed investigations were under- 
taken into the possible causes of the observed differences. 

As an example of the use of the unweighted matched totals, Table 5 presents counts for total 
farm area and categories of land use at the Canada level. The results indicate that less land 
was reported on the census than the survey for all land use items except improved pasture and 
other improved land. Relative differences between census and survey totals were smallest for 
items such as cropland, which are of major economic value and hence are clearly defined and 
seldom misunderstood by farm operators. Items of more marginal agricultural and economic. 
value, however, tended to display greater discrepancies. The largest relative differences were. 
observed for the category of woodland. In order to demonstrate some of the detailed evalua-. 
tion techniques made possible by the matched file, further results of the study of data on 
woodland will be discussed. 
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Table 5 


Comparison of Census and AES-FES Totals for Matched Farms, 
Land Use (thousand of acres), 1981, Canada® 


Item Census Survey Percent 

Total? Total? Difference® 
Total area of farms 13,059 14,091 —7.3 
Improved land 8,798 8,801 -- 
Cropland 6,046 6,167 —1.9 
Improved pasture 804 682 18.0 
Summerfallow ib gAThnL 1,816 —2.1 
Other improved land 170 137 24.3 
Unimproved land 4,261 5,291 — 19.5 
Woodland yes 1,102 — 52.5 
Other unimproved land SR 4,189 — 10.8 


* Excluding Newfoundland, Yukon and Northwest Territories. 

> Records for which census or survey data were entirely imputed have been excluded leaving 16,388 
matched farms. Census and survey totals may not equal the sum of the components due to rounding. 

Census total - Survey total x 100 


Survey Total 


© Percent Difference = 


| 5.5 Detailed Comparisons of the Matched Records 
While the matched totals provided a measure of the overall biases in reporting, they masked 

detailed information on where and why the differences occurred. For instance, was the dif- 
| ference in woodland caused by large discrepancies in only a handful of holdings, or was it con- 
‘sistent across all records? Did the response differences vary in magnitude or direction across 
| types of operations or regions of the country? By comparing census and survey responses at 
the individual record level, the characteristics of the reporting differences were studied in detail. 
| Table 6 presents an example of the type of investigation facilitated by the matched evalua- 
' tion file. Only those holdings for which a non-zero value of woodland was reported on either 
| the census or survey are included. The operations are classified by the size and direction of 
_ the difference between the census and survey values for the item, and cross-classified by the 
- amount of woodland reported on the census. 
_| The table indicates that less woodland was reported on the census than on the survey for 
i the majority of holdings. Differences tended to be small, clustered in the | to SO acres and 51 
- to 150 acres ranges even for operations with large amounts of woodland on the census. Of note 
also were the 3,177 holdings, or 34.3 percent of the universe of interest, for which the census 
ef value of woodland was zero but the survey value was greater than zero. This is in contrast to 
» ‘he 807 holdings, or 8.7 percent of the universe, in which the opposite case of zero acres of 
YY woodland on the survey but greater than zero acres on the census was observed. Examination 
_ f these results suggested that the census and surveys may not have been obtaining measures 
, )f the same quantity, and prompted further study into their collection methodologies and ques- 
, ionnaire formats. 
a On the census questionnaire the respondent was asked to report the area of woodland, with 
_ urther instructions in the census representative’s manual indicating that only land ‘‘with seed- 
» ings or trees which had or would have value as timber, fuelwood, or Chirstmas trees’’ be in- 
\luded. In contrast the AES and FES interviewers instructed respondents to ‘‘include woodlots, 
ut-over land, etc.’’ with no additional instructions that the land be of present or future com- 
oe value. As well, woodland was requested twice on the surveys, once immediately after 
he reporting of total area, and later as a component of land use, and thus received greater 
mphasis. 


| 
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Table 6 
Comparison of Census and AES-FES Responses for Matched Records, 
Difference in Woodland by Census Value of Woodland, Canada’, 1981 


Difference: Census Value of Woodland (acres) 

Census Woodland 0 1 3 10 70 240 400 Total _— Percent 

- Survey Woodland to to to to to or of Re- 
2 9 69 239 399 more porting | 

(acres) Farms 


Number of Reporting Farms? | 


less than — 500 241 0 3 16 16 6 11 293 3.25) 
— 500 to —251 248 0 4 24 21 12 7 316 3.4 | 
—250itoy 151 268 4 1 Di 44 9 4 357 3.9 
—150 to —51 715 5 30 167 145 24 19 1,105 11.9 
—50 to -1 1,705 39) PH} 1,053 370 57 28 3,549 38:30 
0 - 26 165 481 151 21 21 865 93 
1 to 50 - 33) 29 1,288 492 33 30 2,147 2312 
51 to 150 - - - 50 294 45 32 421 4.55) 
151 to 250 - - - - 65 2D 17 107 1.25 
251 to 500 - - = = - 33 42 75 0.8 — 
greater than 500 - - - - - - 36 36 0.4 
Total 335 149 709 3,106 1,598 285 247 Oi 100.0 


4 Excluding Newfoundland, Yukon, and Northwest Territories. 
> Including all operations which reported woodland on either the census or survey. Excluding records for which either | 
the census or survey data were totally imputed due to non-response. | 


As a result of these differences, it was believed that certain areas of woodland of questionable | 
commercial value, which were reported on the survey, may have been excluded from the cen- 
sus. As well, some areas may have been reported on the census under different categories of 
land use, such as other unimproved land. Study continues into these and other hypotheses, us- | 
ing the matched file to investigate possible causes of the observed response differences. 

When summary tabulations from the matched file failed to suggest causes for observed biases, 
the study of individual records which displayed large discrepancies between census and survey 
values for the items of interest was often informative. By comparing census and survey responses | 
for other related items, it was sometimes possible to identify misclassification between categories 
or other causes of reporting differences. | 

Table 7 shows a number of variables for a record with one of the largest differences between 
census and survey values for total farm area. In this case, the discrepancy in total area was 


Table 7 


Census and Survey Recorded Values for a Particular Matched Record 


Total Land Use (acres) Total 
Farm Crop- Improved Summer- Other Wood- Other Cattle: 
Area land Pasture fallow Improved land = Unim- and 
(acres) eand proved Calves 
Land 
Census 
Values —-2,640 1,035 0 1,240 15 0 350 920 | 
Survey 


Values 17,000 1,010 0 970 20 0 15,000 813. 9 
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i 
concentrated in the category of other unimproved land; only minor differences existed between 
responses for cropland, summerfallow, and other improved land. It appears that most of the 
land which was reported as other unimproved on the survey was excluded from the census 
response. Similar studies of other individual records identified other potential discrepancies in 
_ the reporting of unimproved land. Theories developed from these studies of individual cases 
were then tested on the entire file to determine if they might apply in general. 


6. CONCLUSION 


The link between the 1981 census and survey files was a powerful tool for the evaluation 
‘of census coverage and response errors. Errors were known to exist in the survey data which 

were being used as approximations to the true values, and limitations of the linkage operation 
were known to have caused spurious matches and unmatches. However, the matched file was 
‘still a valuable source of data for investigation of quality concerns. Studies based on the record 
level matches broadened the initial results obtained from the macro-comparison of census and 
‘survey estimates. In addition they brought individual problems into focus by allowing detailed 
investigation of particular aspects. 

The evaluation produced valuable results on census undercoverage. Studies based on the un- 
matched survey records showed that the holdings missed by the census were, in general, smaller 
than average in terms of total land area, livestock, and value of agricultural products sold. Thus 
| concrete evidence was provided to support the widely-held theory that the census tended to miss 
| holdings of marginal economic and agricultural value. 

Studies of response differences for land use identified categories in which discrepancies were 
concentrated, tendencies for confusion between certain classes, and variations in differences 
among regions of the country. Possible revisions to questionnaire format and wording, or col- 
lection methods, have been considered as a result of the study. Some of the variations are known 
| to have been caused by difficulties in defining certain land use categories, due to the lack of 
clarity in the concepts themselves. Problems such as these, that result from confusion in the 
| ninds of the respondents as to which land should be reported under which category of usage, 

May never be completely solved. However, the recognition of the existence of a problem, and 
he study of the characteristics of its occurrence and its effect on the data, are very valuable 
contributions to future planning. 

_ The 1981 data quality evaluation had far-reaching effects for the census. In response to its 
nain goal, the study identified quality concerns in the 1981 census data. A publication (Statistics 
Canada 1984) was prepared to provide users with an indication of data quality, and to advise 
hem with respect to particular problems which had a noticeable impact on the data. Looking 
 urther ahead, the evaluation has served as input to the planning of 1986 census procedures. 
3y identifying items for which coverage or response errors occurred in 1981, the study has pro- 
vided a list of areas requiring further consideration of collection and processing methods. 
The impact of the data quality evaluation was not restricted to the Census of Agriculture 
slone. The comparison of census and survey responses also identified problem areas in the other 
_ jlata collection vehicles. Improvements to the National Farm Survey, the annual probability 
urvey which has replaced the AES and FES, may result from the census study. 

A further benefit of the study is the knowledge gained on the use of record linkage for evalua- 
jon purposes. The experience in matching data at the individual record level, using both com- 
cuter and manual means, could provide valuable input to other linkage projects. In particular, 
nowledge of the problems encountered, their causes, characteristics, and possible solutions, 
ould result in improved procedures for other studies. 

In summary, the 1981 Census of Agriculture Data Quality Evaluation project has provided 
urther evidence of the power of matching in the study of non-sampling error. The investiga- 
Ons using matched and unmatched records, and macro and micro-level comparisons, have 
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produced measures of quality of the 1981 census data, and identified items for which error 
have impacted significantly upon the results. As an extension of the original project mandate 
input was provided to the planning of the 1986 and subsequent censuses, by indicating area 
in which further research into possible changes in methodology was required. The evaluatio1 
also identified potential problems in the surveys used for comparison, thereby contributing t 
the planning of future vehicles for the collection of agricultural data. Finally, the study provid 
ed valuable experience and insight into the application of record linkage techniques for dat: 
quality evaluation. 
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Unbiased Estimation of Domain Parameters 
in Sampling without Replacement 


ARIJIT CHAUDHURI AND RAHUL MUKERJEE! 


ABSTRACT 


A finite population of size N is supposed to contain M (unknown) units of a specified category A (say) 
constituting a domain with mean yp. A procedure which involves drawing units using simple random sampling 
without replacement till a preassigned number of members of the domain is reached is proposed. An unbiased 
estimator of p is also derived. This is seen to be superior to the corresponding possibly biased estimator 
‘based on a comparable SRSWOR scheme with a fixed number of draws. The proposed scheme is also shown 
to admit unbiased estimators of M and the domain total 7: 


KEY WORDS: Domain estimation; Simple random sampling without replacement. 


1. INTRODUCTION 


In large scale sample surveys, utilization of available resources and consideration for efficiency 
often demand realization in a sample of adequate representation from a specified category (A, say) 
of members with required characteristics. For example, clients and users of survey data may insist 
on estimates from a sample with a specified (m, say) number: (1) of farmers (i) using a particular 
fertilizer, (ii) employing a particular irrigation and cultivation technique and (iii) ready to respond 
truthfully to queries made; (2) of manufacturers using iron and steel with a specific purpose; (3) 
of household members with a requisite academic qualification, etc. While designing a sampling 
plan for the purpose, in spite of careful efforts, it is often possible that ‘frames’ may not be accurately 
constructed. The faulty list may be supposed to include N units which are well in excess over the M 
_ genuine units of the required A-category. Hence arises a problem of sampling to yield estimators 
_ for the mean, (and also total and size), of the domain of A-members. A solution to this problem 
_ \s attempted below using ‘inverse’ SRSWOR scheme. Inverse sampling plans with replacement are, 
however, available in the literature (vide Haldane 1945, Sampford 1962 among others) for esti- 
mating the proportion f = M/N of domain elements. Domain estimators for are also given by Rao 
_ (1975) but they are ratio estimators and are not unbiased. The proposed inverse SRSWOR scheme 
_ s seen to admit an unbiased estimator of » which is more efficient than the corresponding possibly 
_ diased estimator based on a comparable SRSWOR scheme with the fixed number of draws. 
| 


2. A METHOD OF SAMPLING AND ESTIMATION 


_ The population J, = (1, ..., /, .... N) is supposed to consist of N units labelled ], ..., /, ..., N 
ind valued y,, ..., Vis +» Vy. Of them some M (unknown) units possess certain exclusive features 
 Oconstitute a class or domain, say A. In practice, some idea about M is usually available and 
‘et the parameter space for M be Ml = {r, r+], ..., R}, where r(> 1) and R(< N) are known. 
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Writing Peres yer Sy oi y; values for the M units of A, estimators are required fol 
pe (os y/ M, and ea ES also for M and T= D7 X,, from a sample containing a 
preassigned number, say m (< r), of units of A. The expressions for the variances of these) 
estimators, presented later as functions of m, may be employed for an appropriate choice of m.. 
For convenience, we shall write X,,,, = ... = X, = 0 for the ‘non-A’ units of /,. 

Let units be chosen in successive draws by SRSWOR till exactly m units of A are realized. The) 
number of draws, u, is then a random variable with a probability distribution P,,(+) (say, depen: | 
ding on the unknown parameter M) which is given by 


eae 

m-1 n—-m|] M-m+1 | 

IP IWS) = = say) (m < n< D+ ™m), 2.1)) 

yu =) . ey Sun (say) ¢ ) om 
ida) ) 

where D = N — M. To avoid trivialities, hereafter, we shall make the reasonable assumption that 

m > 2. Then the following results hold for the above inverse sampling scheme. 


Lemma 2.1. Every parametric function /(M) is unbiasedly estimable. 


Proof. Let h(u), if available be an unbiased estimator (UE) of /(M). Then 


D+m 


SM) = © AO) Buy 1S MSR. (2.2) 


If the above system of R — r + 1 equations in N — r + 1 unknowns A(m), ..., h(N — r + m) 
be written in matrix notation, then the fact thatg,,>O0(m< n< D+mr< M< R) im- 
plies that the resulting coefficient matrix is of full row rank. This guarantees the existence of a 
solution and completes the proof. 


Remark. In particular, if R = N then the number of equations in (2.1) equals the number of 
unknowns. As such the coefficient matrix becomes nonsingular and every parametric functigg 
J(M) becomes uniquely unbiasedly estimable. 


Corollary 2.1. A UE of M based on u is M(u) = Nim — 1)/(u — 1) = M (say). 


Proof. First observe that the assumption m > 2 ensures that u > 1 with probability 1 (whatever 
be M) so that M(u) is well defined. Now 


1 7 D+m 1 
ee 
ue M! D! (gpl bases?) (ae) 
(M — m)! (m — 1)! M 7=™ (n— m)! (D — n + m) 
M D! M — m)! (m — 2)! (N — 1)! M 
= ° = — — , VYMEmM 
(M — m)! (m — 1)! M (M — 1)! D! Nim — 1) 


Hence the result. 


Remark. The relation (2.1) and Lemma 2.1 may be employed to find Vi«M) and a UE of this” 
variance. The resulting algebraic expression, although straightforward to evaluate numerically in - 
any practical situation, are somewhat involved and will not be presented here. | 


In the following, S? = (M — 1)! XY (x, — w), gu) and (u) are any UE’s for M™' and M 
respectively (available by (2.2) above) L’ denotes summation over the A-units included in the 
sample, xX = m'L'X,, Z = m'2'X? and s? = (m — 1)! D(X, — XY. | 
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Theorem 2.1. A UE of p is x with V,(X) = S’(¢m@ — Ys). A UE of V.Ax) is given by 
v&%) = s*(m-' — qtu)). 


Proof. Easy and hence omitted. 


Theorem 2.2. (i) A UE of T is T = Mx with 
VAT) = S?(Y%m — Yo) Ey (MP?) + wV,(M). 
(i) (7) = T? — [Lu)(Z — s*) + Ms?] is a UE of V,(T). 
Proof. The proof of (i) is easy and hence omitted. To prove (ii) note that 


E [{e(u(Z — s?) + Ms} |u] 


M 
= fu(M*& X? — S*) + MS? = C(u)(u? — M~'S?) + MS?. 
ence 


E, WT) = E,AT) — (M?q? — M~'S*) + MS") = E,{T?) — P = V,{T). 


3. COMPARISON WITH SRSWOR WITH A FIXED NUMBER OF DRAWS 


In this section, first it will be shown that if one insists on unbiased estimation of p then our 
‘rategy will be superior to the one based on SRSWOR with a fixed number of draws. Second- 
1, this superiority will be demonstrated even when biased estimators are allowed. 

_ Let d be a fixed (somehow) number of draws in SRSWOR sampling, § a sample so drawn, 
‘A the set of A-units in § and C the cardinality of §M A. We will use, for this scheme also 


revious notations P,, E,, V,,to imply phenomena relevant here. Then for such a sampling 
e have: 


t heorem 3.1. , admits a UE if and only ifd > N-—r+ 1. 


roof. letd > N-—r+1.ThenP,, [c = 0] = 0, 7 ME Mand = ey X, is a UE Of p. 
_ To prove the necessity it will be enough to show that if d = N — r, then p does not admit a UE. 
or this the following notations will be used. let /,, ..., 7, be d distinct increasingly ordered units 
at of 1, ..., N, constituting the elements of § and such that some & of them (0 < k < Q@), 
yi, .., i, (increasingly ordered) belong to A. Then we write § = (/,, ..) J), 8) = Gs os YD) 
Seen) (Sosthatdeu= 0 =>18', = rand |k = d-=>4's= $) and X($'). = CX}, 5.2, Xp), a 
quence of X, values for the units in §’. Then if there exists a UE for p, say re we may write 
= 1(X(S')|S) such that 


LENG UG Je aes Se, GRP Come a) cee 6 (3.1) 


i For0 < k< d, lett, = rAx(s')|9, L, being sum over all samples with exactly k A-units. 
“tearly ¢, is free from X;,’s. 
If d=N-r, thn M={N-d, N-d+1,...,N}. Suppose M=N-—d+ J 
~(< j < d). Then the A-units may be chosen in (77) = (,“; ) ways. Accordingly ( \ .) equa- 
_ bns are involved in (3.1). Summing over the number of ways of choosing the A-units, (3.1) yields 


() [v= 1) 7 

: ale 

zy Anty = aS oo ae , (3.2) 
at N-d+t+jJ 


| 
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where, forO < jS w< d,a, = N~4) if N — d > w — J; 0, otherwise. From (3.2) the 


Wes 
solutions for the ¢,’s may be obtained as | 
d\ | 
C= (7 | () Nas d, (3.3)} 


and the validity of (3.3) follows from the fact that 
5 (N-d\{d\_[{ NN! | 
w=] Mi a) va d Ali 


In particular, (3.3) yields t, = N ‘(7) T. But then ¢, is not free from the X;’s implying a con-| 
tradiction, proving the necessity and completing the proof. 

Thus with a fixed size (d) SRSWOR scheme, for unbiased estimation of » we need. 
d > N — r+ 1 which may become too large (especially if r is small) making the scheme opera-. 
tionally inconvenient. Even if d > N — r + 1, the fixed size SRSWOR scheme together with) 
the UE ~# = c 'L'X, can be seen to be less efficient than the strategy described in the 
preceding section when compared at equal level of cost of inspection. | 

To elaborate, suppose d > N — r + 1 and note that 


Vili) = S| B00) — |. G.4) 


For our inverse sampling scheme, by (2.1) the expected number of draws is given by 
m(N + 1)/(M + 1) and, to make our scheme comparable to a fixed size (d) scheme, this should 

equal d i.e. one should have m = d(M + 1)/(N + 1), in which case Theorem 2.1 yields 
N+ 1 

d(M + 1) uf ; 


VA) = = (3.5) 
Since 
Nyt N pried os 


Gra LAG rtierys Weve 


it follows that (3.4) is greater than (3.5), proving our assertion. 

It is also interesting to compare our strategy with the fixed size scheme when a possibly bias- 
ed estimator of is allowed in the latter. In fixed size (d) SRSWOR scheme, consider the usual 
(ratio) estimator of p» given by [vide e.g. Rao (1975)] 


pore icn  aN givet a0 
= 0 ifc =0 


The bias in y* equals —p P,{c = 0) (observe that ifd > N — r + 1, then Pfc = 0) = 0, 
VY M € M and p* reduces to the UE yp defined earlier) and it can be shown that 


MSE,(u*) = S?E (hk — Wi) Pde = a) + wP,Ac = 0). (3.6) 


A straightforward analytic comparison between (3.5) and (3.6) is difficult but as numerical ex- 
amples including the two cited below suggest, in most practical situations (3.5) will be smaller 
than (3.6), indicating the superiority of our strategy even when a possibly biased estimator is. 
allowed in the fixed size scheme. 


Example 3.1. The following data relate to the aggregate percentage of marks of all the students | 
who passed the Bachelor of Statistics Examination of the Indian Statistical Institute (ISI) dur- | 
ing the last five academic years ended 1984!. 


' The data are obtained from the office of the ISI Dean of Studies to whom the authors are grateful for granting | 


: 
| 
an access to them. 
| 
| 
| 


nl 
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68 80 80 TZ 87 71 55 75 85 52 82 

76 73 54 ay) 51 56 48 ON 54 76 69 

87 81 68 74 58 56 71 66 69 81 59 

65 83 79 72 50 44 65 61 =) 50 73 

85 87 64 70 48 58 61 53 56 62 61 

74 62 56 62 58 58 66 70 80 74 80 
Suppose it is desired to estimate from a sample the mean score of those students who obtained a first 
class (i.e. sixty percent or above). Then N = 66, M = 44, » = 73.1818, S* = 61.6871. For a fixed 
size SRSWOR scheme with d = 10, (3.6) equals 9.5967. The comparable m in our inverse sampling 


strategy is dM + 1)/(N + 1) = 6.72 and with this = 6, 7, (3.5) equals 8.8792, 7.4105 and the result- 
ing gains in efficiency, compared to the fixed size scheme, are 8.08 and 29.50 percent respectively. 


Example 3.2. As a somewhat less traditional example, consider the problem of estimating the 
mean of the prime numbers among the first sixty natural numbers. The N = 60, M = 18, » = 24.5, 
S’ = 350.1471. For a fixed size SRSWOR scheme with d = 7, the value of (3.6) is 205.4654. The 
comparable inverse sampling strategy requires m = d(M + 1)/(N + 1) = 2.18 and with this = 2, 
(3.5) equals 155.6209, indicating a gain in efficiency by 32.03 percent. 


4. CONCLUDING REMARKS 


In this paper we have considered the estimation problem for a single domain. In large scale 
surveys estimators are often required for several domains in which case the present procedure may 


_ be modified as follows. 


Let there be ¢ domains, the domain sizes M, being unknown, having respective parameter 
spaces M, = {r,, T,,,, +» R,}, where r, and R, are known (1 < k < 2). Let p, and S? and 


denote the population mean and variance of the study variate in the kth domain. The sampling 


' scheme may be inverse generalized hypergeometric, i.e. inverse SRSWOR may be continued till 
at least m,, m, ..., m,(m, < r, for each k) units of the Ist, 2nd, ..., th domains are realized. For 


each k, clearly the number of units, say &, in the sample from the kth domain is now a ran- 


_ dom variable, with P,fé, = m,) = 1 (where m = (M,, ..., M)', Pyp E,, the corresponding pro- 
_ bability and expectation operator), since even when the quota for kth domain is filled up, sampl- 


ing may have to be continued to fill up those for the other domains thus possibly including in 
the sample some additional units from the kth domain. The mean, x,, of the units in the sam- 


ple from the kth domain is a UE of p,with a variance S [E, (4%) — “J, 1S k <4 


In this set-up also numerical investigations (records omitted since they seem uninteresting in the 
present context) suggest that the inverse sampling strategy will be more efficient than the one based 
on fixed size SRSWOR when compared at the same level of cost. For multidomain situations, 
however, the detailed algebraic expressions become somewhat involved so that an analytic com- 
parison along the line of the preceding section becomes difficult and hence not reported here. 


ACKNOWLEDGEMENT 


The authors are thankful to the referee for his highly constructive suggestions that helped im- 
provement on an earlier draft. 


REFERENCES 


Haldane, J.B.S. (1945). On a method of estimating frequencies. Biometrika, 33, 222-225. 

Rao, J.N.K. (1975). Analytical studies of sample survey data. Survey Methodology, 1, 1-76. 
_Sampford, M.R. (1962). Methods of cluster sampling with and without replacement for clusters of unequal 
_ sizes. Biometrika, 49, 27-40. 


Kt t? ¢ ra ay - 
¥ J i a 
rs “A ° = Wr 
rz ¥) Be) eh 
pag Aap 
nm. 


iF A? ae “ a, a2 ef 

peal ya aA 1 GPP, Ra eh siacus ¢ men) eimpgen a) ah 

ris HALES» auld OR Lye ake ae iene} sme 

wy de ap hg ahs “ROME A) OF PPT & he mulls 

. yp re TE Feet Yitlw bir DMEE VE 
ipa usantie eet Weve ave eo SA a yeti 


at Sige RS Way wig *4 Nie 7 
 amlihete: Bi, See Honing tari ei =a a jen te Pe 
4 i its #£ vel ell Deed) YAle Ae eral ay at Ve 


ih } ; . > : ~ fie ne ae is on ge ITO = 
; f » WAR wh Shiga (ysuate giuleittiin sasvet olde 
ow? Ft LA ortvegeiaiiio tT oles ‘ ysitecvilyat, QOLA CED Strate 


wo Wid ii ee | iy rahi We airy » | 
: ieain a fad ited a rage, tive 


rane) | -«hpup rei aap St fayette san Oty 
, wu anu loyyye? (0) bone py dara hh frome 


ot | evinlloy ae’ Bey 
1 od sii# pinindy 7 pot we bp 
pie | my A ti Aa alt a Ma 
irate aba Wt on] opin Goa sean didllidlbvegoq aed 
my 7 Jpeg sort a wishes sh-"i0 BA “ans 
i i ae] mo? 1% Wana FOF 8 TW a oh ager 
‘ ' qeime cit 1) o d.yet, atteyio. Sota viele S 
' i i Vyeaectgye) Gi Su aks Gees 
we, 41 Waite 91 apt Ty eet CAESAR 
: ww, 1 a 2A aoa cer TTF Gy Genergnad, segs wet etn 
ios Teel iments mls bh 
; hae 1G PTR whi Be a Nba 
; ite + encanta ts wyors te ait 1g THE 
i a a bremenes pe rvertt ovl BAL 105 tel etuieitier 
fi a ni =) oe Terman Llp Hi i t MAN 7 


mi) wiper? PAE nil) htsveonhaaiva) ‘ 
a = 8 od ihe uriy: “5 ery uf! tc) mi Atty *srgle 
2 ; j by } 


te 2UNDETT TRIO AS 1) 41 Uh it ae aa 


‘mest, a 24 L ounce (3.3) wal? Ge 
vii nis Gt aghaaion ily At atwarad dan, rye A 
mk 17) gene Ge 


pal dae ie oe aaah cP ld 
oe eR TG (ie AGA Gsaravricel tantienial 
1G lei py aid eo To badleaay J ny eae) 21S 
’ # gern’ udh enbotd ethtae We ellain tolnihins RRR) 
ny! sah limpet athe chy ead x nandeona a te rth 
/ is 12 alvin 
am & ih 


Survey Methodology, December 1984 187 
Vol. 10, No. 2, pp. 187-197 
Statistics Canada 


A Methodology for Surveying Disabled Persons using a 
Supplement to the Labour Force Survey! 


D. DOLSON, P. GILES, and J.-P. MORIN2 


ABSTRACT 


In response to a need for data on disabled persons in Canada, Statistics Canada undertook a program 
to create a disability database. This includes using supplements to the Canadian Labour Force Survey 
in the Fall of 1983 and the Spring of 1984, as well as including questions on the 1986 Census of Popula- 
tion. A general discussion of the background and content of the survey is presented. A comparison of 
screening methodologies conducted by Statistics Canada in November 1982 and January 1983 is presented 
and the results are compared. 


KEY WORDS: Disability; Screening; Activities of Daily Living. 


1. INTRODUCTION 


On May 23, 1980 the Canadian government created the Special Parliamentary Committee 
on the Disabled and the Handicapped. In February 1981 (the International Year of the Disabl- 
ed) this committee published its report, entitled ‘‘Obstacles’’ [4]. Recommendation 113 of the 
“Obstacles’’ report reads in part: 

“That the Federal Government directs Statistics Canada to give a high priority to the develop- 
‘ment and implementation of a long-term strategy which will generate comprehensive data on 
_ disabled persons in Canada, using population-based surveys and program data.’’ 

The government, wishing to respond positively to the recommendations contained in the 
_ report, thus requested Statistics Canada to undertake a survey of disabled persons. 

This paper focuses on disability surveys conducted as supplements to the Canadian Labour 
Force Survey (LFS) in October 1983 and June 1984 and on tests which were done in November 
1982 and January 1983. 


2. DEFINITIONS 


Definitions developed by the World Health Organization (W.H.O.), given in McWhinnie 
(1980), were employed by the Special Parliamentary Committee. These definitions arise out 
of a model which focuses on the consequence of disease, and addresses the following illness- 
related phenomena. 


DISEASE —> IMPAIRMENT - > _ DISABILITY —*» HANDICAP 


This paper is a combined version of the two papers entitled ‘‘A Methodology for Surveying Disabled Persons Using 

a Supplement to the Canadian Labour Force Survey’’ by P. Giles and D. Dolson, and “‘The Canadian Experience 

with Screening for Disabled Persons in a Household Survey”’ by P. Giles, D. Dolson and J.-P. Morin. These papers 
| were presented at the 1983 ASA meetings in Toronto. 


P. Giles, Business Survey Methods Division, D. Dolson and J.-P. Morin Institutional & Agriculture Survey Methods 
| Division, Statistics Canada, Tunney’s Pasture, Ottawa, Ontario, Canada KIA OT6. 
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As given in World Health Organization (1980), the definitions of these terms are as follows. 


Impairment: In the context of the health experience, it is any loss or abnormality of 
psychological, physiological or anatomical structure or function. 

It is characterized by losses or abnormalities that may be temporary or permanent, and that 
include the existence of an anomaly, defect, or a loss in a limb, organ, tissue, or other structure 
of the body, including the systems of mental function. Impairment represents the exterioriza- 
tion of a pathological state, and in principle reflects disturbances at the level of the organ. 


Disability: It is any restriction or lack of ability (resulting from an impairment) to perform 
an activity in the manner or in the range considered normal for a human being. 

It is characterized by excesses or deficiencies of customarily expected activity, and may be 
temporary or permanent, reversible or irreversible, and progressive or regressive. Disabilities 
may arise as a direct consequence of impairment or as a response by the individual, particularly 
psychologically, to a physical, sensory, or other impairment. Disability represents the objec-. 
tification of an impairment, and as such, it reflects disturbances at the level of the person. 


Handicap: It is a disadvantage for a given individual, resulting from an impairment or disabili- 
ty, that limits or prevents the fulfillment of a role that is normal (depending on age, sex, and 
social and cultural factors) for that individual. 

Handicap is concerned with the value attached to an individual’s situation or experience when 
it departs from the norm. It is characterized by a discordance between the individual’s perfor- 
mance or status and the expectations of the individual himself or the particular group of which 
he is a member. Handicap thus represents the socialization of the impairment or disability, and 
as such reflects the consequences for the individual (cultural, social, economic and evr On a 
that stem from the presence of impairment and disability. 

To explain these definitions more clearly, consider an example: 


DISEASE Arthritis 


Y 


IMPAIRMENT _ Inflammation of the joints in 
‘i hands and fingers 


DISABILITY Inability to grasp and hold 


Y 


HANDICAP Limited in the kind or amount 
of work which can be done 


3. TARGET POPULATION 


Ideally the target population would include all persons in Canada who are disabled according 
to the above definition and subject to constraints on severity and duration. Severity can be regard- 
ed in terms of the person; i.e., how severely is a person disabled, or in terms of disability; i.e., 
how severe is the specific disability. In defining a target population, a measurement of severity 
must be included, if only implicitly. The duration of disability must be explicitly addressed. 
To capture all disabilities, including those arising out of acute illnesses of relatively limited dura- 
tion, would identify a large percentage of people and run contrary to the spirit of the ‘“‘Obstacles’ ; 
recommendation. Nevertheless, to limit the population to the permanently disabled is avoiding 
the issue and ignoring the needs of the long-term but not chronically disabled. 


i 


| 
. 
) 


; 
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As explained in the next section, the data were collected through the use of the supplemen- 
tary capacity of the LFS. In addition to the normal constraints of the LFS, one significant limita- 
tion was imposed by the use of the LFS. The target population of the disability survey was 
not to include the ‘‘mentally ill’’. For example, the target population excluded illness such as 
amnesia, neuroses and phobias but included impairments of intelligence such as mental retar- 

_ dation or dyslexia. It was felt that asking for this information could be very sensitive in nature 
and negative reactions could compromise the primary objectives of the LFS. 

Thus the target population for the disability survey tests includes all persons having one or 
more physical (nonbehavioural) disabilities, or knowledge acquisition or other educational 

| disabilities (arising from impairments in intelligence, attention, psychomotor functions and 
' language), whose duration has been or is expected to be at least six months. It also includes 
| individuals suffering from diseases of a chronic and degenerative nature and which have a high 
_ probability of producing impairments which are physically disabling. In addition the normal 
constraints of the LFS are in effect which precludes individuals in institutions. 


4. DATA COLLECTION 


The difficulty is in translating the definitions into a set of questions which identify persons 
of interest from a set of persons in the general population. This leads to setting an objective 
to collect information on those who have a high probability of being disabled by any user’s 

definition, and at the same time, keeping the number of people surveyed within reasonable 
limits. 

The first option studied was to include questions on the Census of Population. However 
the diverse data requirements would have required ten to thirty additional questions, which 
was clearly not possible. The second option considered was to include a limited number of ques- 
tions on the Census of Population, which would identify the disabled. In order to meet the 
data requirements, a follow-up survey would be required. In fact, this option has been approv- 
ed and disability questions will be included on the 1986 Census questionnaire. This will provide 
detailed estimates for small areas. However, results from the follow-up survey will not be 
available before 1988 or 1989. Given that the current demand for data was high as well as the 
fact that this demand was mainly for national baseline estimates, the Census/follow-up option 
was considered inadequate by itself. 

For meeting current data requirements, two alternatives were considered. The first possibli- 
ty would be to mount on a continuous or periodic basis a household survey similar to the Canada 
Health Survey, to provide a profile of the disabled and handicapped. Such a survey could use 
survey methods designed particularly for the collection of disability data. However, resource 
2onstraints made this proposal not feasible. The second alternative was to use the supplemen- 
lary capacity of the LFS. For reasons of expediency and cost, this was the method chosen. This 
das the secondary advantage that the resulting disability data on the individual can be directly 
‘inked to their labour force data collected by the LFS. 

The data collection for the disability survey was conducted in two stages. First, all persons 
all households in the LFS sample, except the one-sixth of the sample which is in its first month 
of the survey, underwent a ’’screening’’ process. Persons of potential interest were identified 

_ »y means of a ’’screening’’ questionnaire. The mode of data collection was the same as for 


- ‘he LFS interview. However, the interviewers were asked to obtain non-proxy interviews as often 


ls possible, even if it meant calling back at a later time. This ’’screened in’’ population was 
hen asked another set of questions in a follow-up survey. All of these interviews took place 
‘bout a week after the screening interview. They were all personal interviews and non-proxy 
esponses only were accepted. This second set of questions was designed to collect the data iden- 
ified as being desirable by a consultation process with the users. 
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The schedule for the survey was as follows. Three proposed screening questionnaires were 
tested in November 1982 and January 1983. More details on these surveys will be given later. 
Based on the results of these surveys, one screening questionnaire was developed. Two ‘‘full’’ 
surveys with a screen and a more detailed questionnaire as described above were conducted 
in October 1983 and in June 1984. 


5. APPROACHES TO SCREENING - OTHER SURVEYS 


The first step in constructing a set of screening questions was to investigate experiences en- — 
countered by other groups that had previously conducted disability surveys. | 

The one approach that has been used in many surveys is the Activities of Daily Living — 
(ADL) approach. The Activities of Daily Living are a set of activities which any person is — 
required to perform during the course of their regular living pattern. Although there is no 
generally recognized ‘‘best’’ set of activities that should be used, the set developed in 1978 
by the Organization for Economic and Co-operative Development (OECD) and noted in | 
McWhinnie (1980) has been used by surveys in several countries; see Klaukka (1981), Mizrahi | 
and Mizrahi (1981), Raymond, Christe and Clemence (1981), Van Sonsbeek (1981), Wilson and 
McNeil (1981). | 

Since a person’s ability to perform an ADL may depend on their use of a physical aid, such 
as an artifical limb, the use of a list of physical aids for screening could be appropriate. 

Another approach for screening is that of major activity limitation. If a person is limited 
in his/her major activity (i.e., work, school, home) that person is probably experiencing some 
disability. This approach has been used in the United States in a pretest for a disability survey | 
(1980) and in the annual Health Interview Survey, and in Canada in the Canada Health Survey 
(1978-79). 

A list of chronic conditions could be useful for screening since persons with chronic condi- 
tions are in the target population but may be missed by ADL’s or activity limitation if the per- 
son has intermittent difficulty. 

Finally a person could be asked a single self-perception question such as ‘‘Do you have any 
physical disabilities or handicaps?’’. 


6. TEST OF SCREENING MECHANISMS 


The three Statistics Canada screening tests used combinations of these approaches. Also per- 
sons aged 15 or over were administered a different questionnaire than those under 15 years 
of age. No suitable set of ADL’s has been compiled for children. In fact, most disability surveys 
that have been previously conducted have excluded children. Here, we will consider only per- | 
sons aged 15 years and older. 

In the November 1983 Labour Force Survey, each respondent was asked ‘‘Does ... now have 
any disability or handicap which has lasted or is expected to last six months or more?’ This 
was called Test 1. Persons screened in by this question were those responding ‘‘yes’’. 

The other approaches to screening were tested using two different questionnaires each ad-‘ 
ministered in the January 1983 Labour Force Survey. These questionnaires were called Test 
2 and Test 3. 

Test 2 included the following sections: a list of special aids, a list of ADL’s, and the activity | 
limitation question ‘‘Are there any (other) conditions or health problems that now prevent or. 
limit ... when carrying out his/her normal daily activities at a job, in school, or in the home? | 
Please report only difficulties which are expected to last more than six months’’. Persons who 
reported using at least one of the special aids or having trouble doing at least one of the ADL’s | 
or who answered ‘‘yes’’ to the above question were screened in. | 
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Test 3 included a list of ADU, a list of chronic conditions, and the following two work disability 
questions: “Is .. . limited in the kind or amount of work he/she can do at his/her job or business 
because of a long-term physical condition or health problem?” (asked only to employed per- 
sons) and “Is ... prevented or limited in the kind or amount of work he/she could do at any 
job or business because of a long-term physical condition or health problem?”. Persons who 
reported having trouble doing at least one of the ADL’s or who had at least one of the chronic 
conditions or who replied ”yes” to either of the above two questions were screened in. 

Lists of the special aids, the chronic conditions, and the ADL’s used in these tests are given 
in the appendix. It should be noted that the ADL’s used in Test 2 and Test 3 were identical 
although slightly modified from the OECD list. In addition, each test takes a different approach 
to the ADL list. Test 2 permits the use of aids to perform the activities, while Test 3 does not. 

These proposed screening methods do not permit an assessment of whether or not the target 
population is being correctly identified, unless they are used on a control population. This has 
not been done for the Canadian survey. 

Test 1 was administered to all persons in households in the November 1982 Labour Force Survey. 
In January 1983, two rotation groups were used for each of Test 2 and Test 3. These rotation 
groups were chosen so that each in-sample household had also been in-sample in November 
1982. This facilitated the comparison of the results of Test 1 with those of Test 2 and Test 3. 


6.1 Major Findings of the Pilot Tests 


The major goal in the analysis of the tests was to determine the set of questions that would 
be most effective in screening in those persons who belong to the target population. Another 
important factor was to determine an effective screen that would also not unduly increase respon- 
dent burden or cost. In addition, sources of non-sampling errors that became evident during 
the analysis were noted so that, for example, survey procedures or questionnaire design could 
be changed appropriately. The following discussion presents the major findings of the analysis. 

The sample size for Test 1 was about 115,000 persons. For each of Test 2 and Test 3 the 
sample size was almost 38,000 persons. For all three tests the samples were about 49% male 
and 51% female. 

Table 1 shows by sex the percentage of the sample screened in by Test 1, Test 2, Test 3 and 
by each section of Test 2 and Test 3. Most notable in this table is that Test 1 screened in only 
5.6% of the sample as compared to about 16% of the sample for either of the ADL questions. 
Given that functional limitation as measured by the activities of daily living is a key indicator 
of disability, this shows that the single question asked in Test 1 is not effective for screening 


_ in the entire target population. 


Table 1 


Percent Screened in by Each Section of 
Each Questionnaire, by Sex 


nee er ya nS SSSIESSISE REESE 


Percent Screened in 


Section 

Male Female 
Their ake 0) Uae Set 0R oe Cand Bo eR See eS See ee 525 aay 
TIAN PANO EY DI eee ae ohn G1, aE ieee pedi PRAT 14,7 16.2 
esto AlOswre ats. cre cil: cols Seve begs 32 2.9 
este SmeACTIVITVmLIMITALONI ES gel, yen sys masts okeiree 5.9 6.1 
Se RO oe I Fe een Se er, were ee 18.3 19.4 
ALS OPN DIE peg ne Ree ne en ae 15,4 16,7 
ESTAS SN VOT Ka ISA DULY icon sous artes ce susngun 13.0 13,1 
Testes en CHronics COUGITIONS «cuss os osc er 25.6 21.4 


AWSRE SS ngs ae eR eee 29.8 31.2 
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Figure 1. Percent Screened in by Test 2 and Test 3 


Percentages of the sample screened in by Test 2 and by Test 3 are shown by age in Figure 
1. For both tests, the probability of being screened in is highly related to age. The probability 
of being screened in is an increasing function of age. Although the degree varies, the same rela- 
tionship holds for each of the Sections of Test 2 and Test 3, for each of the seventeen ADL’s 
and for most of the chronic conditions (multiple sclerosis, epilepsy, cerebral palsy, cystic fibrosis 
and muscular dystrophy are exceptions). 

Since functional limitation is a good indicator of disability the question for the analysis was 
how to include the ADL’s on the screen rather than whether to include them. The ‘‘with special 
aid approach”’ (Test 2) selected 15.5% of the sample while the ‘‘without special aid approach”’ 
(Test 3) selected 16.0% of the sample. The difference between these figures is not statistically ' 
significant. For both respondents and interviewers the ‘‘with special aid’’ concept seemed to 
be more natural and more easily understood and was therefore chosen for the finalized screen. 

Since the complete set of activities can be used to obtain a measure of degree of disability, 
all seventeen ADL’s were retained for the finalized screen. | 

The special aids section in Test 2 screened in 3.0% of the sample. Of these, 84.2% were | 
screened in by the ADL section. Thus although this type of data is of interest for the disability — 
database, the aids section is not an efficient screening mechanism, especially in combination i 
with the ADL section. Consequently, questions on aids are not included on the finalized screen. | | 

| 
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In addition to activities of daily living, major activity limitation is also an important aspect 

of disability. The Test 3 questionnaire addressed this by the two questions noted earlier in this 

section. These questions, however, considered it only from the point of view of work disability. 

_Of the persons in the Test 3 sample, 13.1% were screened in by these questions (6.8% of employed 
persons, 6.6% of unemployed persons and 22.7% of persons not in the labour force). 

Although there were no obvious problems with the data from the work disability questions, 

there were some operational difficulties. In particular, the questions sometimes seemed irrele- 
vant to retired persons. Thus, the finalized major activity limitation question was adapted in 
order to better suit persons not in the labour force. 

The chronic conditions section in Test 3 screened in 23.9% of the Test 3 sample. Of these, 
37.9% were not otherwise screened in. There were two main problems with this section. First, 
ithe question was difficult to answer for respondents who were not sure of the nature of their 

particular condition(s). Another difficulty is that the data sought on the follow-up question- 
naire, are generally more pertinent to persons who are currently disabled. Thus persons screen- 
ed in by a chronic condition, but not currently having a functional limitation or a major activi- 
ty limitation would be interviewed for the follow-up and probably provide little useful data. 
| Given these problems, chronic conditions are not used as screening criteria on the finalized 
screen. One exception to this is mental handicap. This one condition is retained as a screening 
item since there may be persons with mental handicaps who are not screened in by the ADL’s, 
or even by the major activity limitation question. 


The project team felt that there would likely be differences in proxy and non-proxy responses 


‘elated to any particular person. To increase non-proxy response, interviewers were instructed 

chat whenever possible the questionnaire was to be completed by interviewing the individual 
_ owhom it applied. If a knowledgeable household member insisted upon responding for other 
- 10usehold members, then this response was to be accepted; although the practice was to be 
- iwoided. 

_ The level of proxy response obtained is considered to be fairly low (20.7% for females, 32.2% 
‘or males). Even after accounting for age-sex differences, it was found that proxy respondents 
_ vere slightly less likely to be screened in than non-proxy respondents. Two reasons can be sug- 
sested as to why the probabilities of selection differ. First, persons who are unavailable and 
or whom proxy responses were provided may be less likely to be disabled. Second, proxy 
_‘espondents may be less likely to state that a person has trouble doing an ADL or a major ac- 

ivity than the person himself/herself. 


7. DATA REQUIREMENTS 


As a result of a solicitation of data requirements from users, 173 responses were received 
vhich identified 588 issues of data needs. The following eleven areas were identified. 


Number of users 


| Issue requesting data 
7 1. Nature of impairment 123 
2. Demographic characteristics 95 
; 3. Employment 85 
i 4. Assistance 77 
<a 5. Education 50 
\ 6. Accommodation 45 
7. Economic Characteristics 41 
P| 8. Transportation 29 
; 9. Social activities 26 
7 10. Health 9 
: 11. Communication 8 


194 Dolson, Giles and Morin: Surveying Disabled Persons 


The nature of impairment/disability/handicap is basic to the survey. Considerable detail is 
collected, including cause of disability. Most users of the data are interested in focusing on 
the impairment or disability groupings separately. Demographic characteristics are always im- 
portant data as they allow the user to identify sectors of the population falling into different 
categories. 

It can be easily understood that employment data about the disabled would be an important 
issue, as employment is a key component to the independent living of a disabled person. A 
great deal of employment data are already collected by the LFS. The follow-up survey will col- 
lect data related to employment limitations experienced as a result of the disability. In addition 
to the analysis of the data for the disabled population, these data will permit comparisons of 
the labour market characteristics of the disabled with those of the Canadian population as a 
whole. 

Three aspects of assistance are considered: technical aids and skills, employment related 
assistance, and education related assistance. In all three areas, need for aids or assistance was 
deemed more important than was use. Under technical aids and special skills, interest is greatest 
for those aids and skills which are most prevalent, or for which special services or facilities 
must be provided. The aids would be grouped under hearing, speaking, seeing and mobility. 
Employment related assistance refers to the impact of aids on the ability of the disabled to work. 

The LFS already determines the highest level of education achieved by each respondent. In 
addition, the follow-up survey will collect data on current educational activity and the impact 
of disability on current and past education. 

The LFS collects information on the dwellings of the respondents. Additional accommoda- 
tion data will be collected on special architectural/structural features, both inside and outside 
the home and other buildings. | 

Economic characteristics will be considered in the following areas: personal income including 
financial assistance received due to disability, sources of financial assistance, and special ex- 
penses incurred as a result of the disability. 

Transportation data will be collected on three types of travel: travel to work or school, other 
local travel and long distance travel. Details on each area will identify the modes of transporta- 
tion used, frequency of use and problems encountered due to the disability. 

Although some interest was expressed by users in data on social/leisure activities, health anc 
communication, no data will be collected for these issues by the present survey. For the firsi 
and third of these issues it was felt that reliable and useful data could not be collected in thi 
survey. Questions related to health are also not included because of the already substantia. 
response burden imposed by issues of higher priority. 


8. RELIABILITY OF ESTIMATES FROM THE DISABILITY SURVEY 


When determining the content of a questionnaire, consideration must be given to the reliability 
of estimates produced for the various data items. It is useless to collect data which will no 
be reliable enough to publish, even if the data requirement has a high priority. The reliability 
of an estimate is tied directly to the sample size. For this survey, the number of persons receiv 
ing a screening questionnaire is fixed. Therefore the reliability of the estimates produced wil! 
depend on the number of disabled falling into the sample and the prevalence rates of eacl 
characteristic of interest. Based on population projections from the 1981 Census of Populatior 
and certain assumptions it is possible to estimate minimum prevalance rates required to pro. 
duce an estimate which is ‘‘reliable enough”’ to publish. An estimate whose coefficient of varia 
tion is less than or equal to 16.5% is considered releasable without qualification by LFS. Tabli 
2 displays the expected minimum releaseable estimates for the disability survey. Estimates 0) 
this size or higher will have coefficients of variation of less than 16.5%, subject to the validit; 
of the following assumptions. 


{i 
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(1) All LFS sampled households are administered the screening questionnaire except the one-sixth 
of the sample which is in its first month of the survey, 


_ (2) 2.95 persons per household on average, 


(3) 5% LFS non-response rate, 


(4) 5% disability survey non-response rate, 


| (5) Design effect of 2.5 (this accounts for the fact that a simple random sample design was 


not used), 


(6) 19% of total adult population and 8% of total child population (aged less than 15) are 


screened in. 


To explain the table in more detail, consider, for example the province of Newfoundland. 
/An estimate of 9,000 persons possessing a particular characteristic will have a coefficient of 
‘variation less than 16.5% and is publishable whereas an estimate of 7,000 will have a coeffi- 
cient of variation greater than 16.5% and is not publishable. An estimate of 8,000 is approx- 
imately 1.4% of the population of Newfoundland. Given the assumptions about percentage 
‘disabled in the population, an estimate of 8,000 is approximately 10.1% of the adult disabled 
population and 59.1% of the child disabled population of Newfoundland. 

The design effects observed from Test 2, Test 3 and the October 1983 Disability Survey for 
number of persons screened in were about 1.5. This suggests that design effects for number 


of screened in persons with specified characteristics are probably also much less than 2.5. 


In the October 1983 Disability Survey 12.9% of adults and 4.8% of children in the sample 
were screened in. 


Table 2 


The expected minimum releaseable estimates 


Province/Region Min P# Min X° Min D* 
Adults Children 
PA tlantaCh Aree ae Sores ee 0.4 7,500 as 16.2 
NI) Ge) Bt BAS eee eS 1.4 8,000 10.1 59.1 
| EH be Fl errencia aenee Bae 2.9 3,500 20.0 > 100 
INR iol oo trons enter Meee Hell 8,500 6.9 54.5 
| SS Sie eee 1.0 7,000 6.7 49.5 
OVHE Se. 6% 5 kao a momo mes 0.5 30,500 333 23.2 
| ONTARIO cree craica ears wha eacrs 0.4 32,500 2.6 220 
PAI eSawaan eto es nok 0.3 10,000 Ley 12.4 
NTAUN GE yee ete sare 0.9 9,000 7.0 47.6 
SVNS) 2h 5%, Soin Ot mete 0.8 7,000 5.0 38.0 
STA rcgius ite chs averala 0.6 13,000 4.1 29.9 
British Columbia ....... 0.7 17,500 4.4 38.2 
a COA eee 0.1 18,000 0.6 4.2 


4 Min P = minimum estimable percentage of the total population, 
> Min X = minimum estimable total, 
| ©‘ Min D = minimum estimable percentage of disabled adults or children. 
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APPENDIX 


Special Aids 
Does . . . now use 
a wheelchair? 
crutches or other walking aids? 
any kind of brace excluding braces for teeth? 
medically prescribed orthopedic shoes? 
artifical limb(s)? 
a hearing aid? 
a guide dog? 
a white cane? 
any other kind of special aid? 


Activities of Daily Liviag 
Does . . . now have any trouble 
walking 400 metres without resting (about 3 city blocks)? 
walking up and down a flight of stairs? 
carrying an object of 5 kg. 10 metres (e.g. carrying a 12 lb. bag of groceries 30 ft.)? 
moving from one room to another? 
standing for long periods of time (e.g. more than 20 minutes)? 
when standing, bending down and picking up an object from the floor (e.g. a shoe)? 
dressing and undressing himself/herself? 
getting in and out of bed? 
cutting own toenails? 
using fingers to grasp or handle? 
reading? 
cutting own food? 
reading ordinary newsprint (with glasses if normally worn)? 
seeing clearly the face of someone from 4 metres (e.g. across a room) 
(with glasses if normally worn)? 
hearing what is said in a normal conversation with one other person? 
hearing what is said in a normal conversation with at least two other persons? 
speaking and being understood? 


Chronic Conditions | 
Which, if any, of these long term conditions or health problems does . . . presently have? 

heart disease 

kidney disease 

lung disease 

cancer 

diabetes 

epilepsy 

cerebral palsy 

multiple sclerosis 

cystic fibrosis 

muscular dystrophy 

paralysis of any kind 

arthritis or rheumatism of a serious nature 

high blood pressure 

hearing trouble (uncorrected by aid) 

vision trouble (uncorrected by aid) 

mental handicap 

any missing limb(s) including finger(s) and toe(s) 

any other long-term condition or health problem (please specify) 
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Corrigendum 199 


{ ‘Logistic Regression Analysis of Labour Force Survey Data’ by S. Kumar and J.N.K. Rao, 
Survey Methodology (1984), 10, 62-81. 


In the formula (26) for the projection matrix M = J — H, the matrix V, should have been 
given as 


diag [p,0 Ey) (nw,), at) pA =p) (nw,)] 


instead of diag [f,(1 —f,)/(nw,), ..., 81 —f)/(nw,)]. Because of this error, the figures 3 and 
4, based on the diagonal elements m,, and h, of M and H respectively, are incorrect. The cor- 
jrected figures are given here as figures 3* and 4* which indicate that cells numbered 2, 3 and 
55 warrant further examination. It may be noted that the diagnostics for assessing the impact 
{of extreme points on the fit (see figures 5-10, pp. 79-81) have also identified cells 2 and 3 as 
extreme points, excepting figure 8. The overall observation in the paper that cells 2, 3 and 7 
may be possible candidates for deletion is unaffected by the above error. 
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GUIDELINES FOR MANUSCRIPTS 


Before having a manuscript typed for submission, please examine a recent issue (Vol. 10, No. 2 
and onward) of Survey Methodology as a guide and note particularly the following points: 


patient: 


Layout 


Manuscripts should be typed on white bond paper of standard size (8% x 11 inch), one 
side only, entirely double spaced with margins of at least 1% inches on all sides. 

The manuscripts should be divided into numbered sections with suitable verbal titles. 
The name and address of each author should be given as a footnote on the first page 
of the manuscript. 

Acknowledgements should appear at the end of the text. 

Any appendix should be placed after the acknowledgements but before the list of references. 


Abstract 


The manuscript should begin with an abstract consisting of one paragraph followed by 
three to six key words. Avoid mathematical expressions in the abstract. 


Style 


Avoid footnotes, abbreviations, and acronyms. 

Mathematical symbols will be italicized unless specified otherwise except for functional 
symbols such as “exp(-)” and “log(-)’’, etc. 

Short formulae should be left in the text but everything in the text should fit in single 
spacing. Long and important equations should be separated from the text and numbered 
consecutively with arabic numerals on the right if they are to be referred to later. 
Write fractions in the text using a solidus. 

Distinguish between ambiguous characters, (e.g., w, w; 0, O, 0; I, 1). 

Italics are used for emphasis. Indicate italics by underlining on the manuscript. 


Figures and Tables 


All figures and tables should be numbered consecutively with arabic numerals, with titles 
which are as nearly self explanatory as possible, at the bottom for figures and at the top 
for tables. 

They should be put on separate pages with an indication of their appropriate placement 
in the text. (Normally they should appear near where they are first referred to). 


References 


References in the text should be cited with authors’ names and the date of publication. 
If part of a reference is cited, indicate after the reference, e.g., Cochran (1977, p. 164). 
The list of references at the end of the manuscript should be arranged alphabetically and 
for the same author chronologically. Distinguish publications of the same author in the 
same year by attaching a, b, c to the year of publication. Journal titles should not be ab- 
breviated. Follow the same format used in recent issues. 
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